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Abstract 

The  current  research  contributed  to  the  study  of  Group  Decision  Support  Systems  (GDSS) 
in  field  experiments  which  involve  real  managers  making  real  world  decisions.  An  independent 
research  effort  conducted  by  the  Air  Force  Institute  of  Technology  in  cooperation  with  the  Air 
Force  Armstrong  Laboratory  evaluated  the  efficacy  of  using  GDSS  technology  in  an  Air  Force 
decision  making  environment.  The  study  involved  middle  management  Air  Force  civilian  and 
military  personnel  who  met  in  small  five-member  groups  in  a  face-to-face  setting.  These  groups 
participated  in  a  choice-task  decision  making  process  over  a  two-day  period.  This  study  compared 
facilitated  GDSS,  facilitated  manual,  and  traditional  unfacilitated  manual  groups  using  a  multi- 
methodological  approach. 

This  study  focused  on  performance  and  secondary  outcomes  resulting  from  GDSS  effects 
versus  process  structure  effects.  The  research  evaluated  two  performance  outcomes:  the  efficiency 
of  GDSS  in  terms  of  decision  speed  and  the  effectiveness  of  GDSS  in  terms  of  consensus.  Post¬ 
process  questionnaires  were  used  to  compare  groups  with  respect  to  eight  secondary  outcomes: 
user  satisfaction  with  the  decision  making  process  and  the  group’s  results,  and  user  perceptions  of 
task  difficulty,  the  group’s  ability  to  remain  task-focused,  the  group’s  productivity  in  terms  of  time 
efficiency,  the  group’s  ability  to  reach  consensus,  the  group’s  level  of  conflict,  the  level  of  process 
structure. 

The  study  concluded  that  GDSS,  process  structure,  and  group  dynamics  impacted  group 
performance.  GDSS  groups  experienced  slower  decision  speeds  and  lower  levels  of  consensus  when 
compared  to  facilitated  manual  groups  which,  in  turn,  experienced  lower  decision  speeds  and  levels 
of  consensus  when  compared  to  unfacilitated  manual  groups.  The  study  showed  that  when  all  groups 
were  considered,  there  were  no  significant  differences  between  GDSS  and  facilitated  manual  groups, 
nor  between  facilitated  manual  and  unfacilitated  manual  groups  with  respect  to  the  secondary 
outcomes.  However,  when  groups  which  received  two  experimental  conditions  were  compared  (i.e., 
cross-over  groups),  the  study  showed  that  participants  perceived  the  tasks  as  being  more  difficult 
using  GDSS  than  during  the  facilitated  manual  process.  Likewise,  participants  perceived  that  the 
facilitated  manual  process  was  more  difficult,  more  structured,  and  led  to  higher  levels  of  conflict 
when  compared  to  the  unfacilitated  manual  process.  These  findings  are  consistent  with  other  field 
studies  which  concluded  that  small  groups  or  groups  performing  less  complex  choice-type  tasks  may 
not  benefit  from  use  of  a  GDSS. 
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THE  EFFICACY  OF  GROUP  DECISION  SUPPORT  SYSTEMS: 
A  FIELD  EXPERIMENT  TO  EVALUATE  IMPACTS  ON 
AIR  FORCE  DECISION  MAKERS 


I.  Introduction 

Over  the  past  two  decades,  advances  in  small  computer  technologies,  particularly  Decision 
Support  System  technologies,  have  steadily  provided  a  means  of  improving  both  the  efficiency  and 
effectiveness  of  individual  managerial  decision  making.  The  use  of  small  computers  to  facilitate 
group  decision  making  is  now  being  evaluated  extensively  in  laboratory  settings,  but  more  impor¬ 
tantly  business  settings.  These  computer  systems  are  called  Group  Decision  Support  Systems 
(GDSSs).1 

1.1  Background 

Efraim  Turban  describes  a  GDSS  as  “an  interactive,  computer-based  system  that  facilitates 
the  solution  of  unstructured  problems  by  a  set  of  decision  makers  working  together  as  a  group” 
(38:132),  (8:589). 2  Several  major  U.S.  educational  institutions,  such  as  the  University  of  Minnesota, 
the  University  of  Arizona,  and  Indiana  University  have  conducted  a  variety  of  GDSS-related  ex¬ 
periments  over  the  past  decade.  Allen  Dennis  and  Brent  Gallupe  discuss  the  evolution  of  DSS  and 
GDSS  research  into  a  new  discipline  within  the  Management  Information  Systems  (MIS)  field  as 
summarized  below  (6:59-68): 

•  Phase  I,  Roots  (1970s):  Computer  messaging  and  individual  DSS  studies. 

•  Phase  II,  Initial  Explorations  (Early  1980s):  Rudimentary  studies  concerning  the  impacts  of 
GDSS  on  group  outcomes  and  processes. 

•  Phase  III,  Early  Experiments  (Mid-to-Late  1980s):  Comparison  of  GDSS-supported  versus 
non-GDSS-supported  groups. 

•  Phase  IV,  Field  Studies  (Present):  Research  to  examine  impacts  on  organizations  in  a  real 
world  environment. 

•  Phase  V,  In-Depth  Studies  (Future):  Examine  a  particular  aspect  of  GDSS  technology — a 
specific  tool  in  a  specific  situation. 


1  Current  literature  includes  GDSS  under  a  broader  category  called  GSS — Group  Support  Systems.  These  two 
terms  are  used  interchangeably  in  this  document  to  more  accurately  reflect  information  cited  from  the  literature. 

2  Turban  modified  a  DeSanctis  and  Gallupe  definition  in  describing  a  GDSS. 
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Although  the  early  research  in  Phases  I  and  II  was  illuminating,  the  critical  findings  in  GDSS 
have  resulted  from  more  recent  studies.  The  focus  of  this  research  effort,  therefore,  was  that  of 
Phase  III  and  beyond. 

The  laboratory  studies  of  Phase  III  primarily  chose  undergraduate  Business  and  MIS  majors 
who  had  little  job  experience  as  participants.  These  studies  usually  involved  a  small  group  decision 
making  task  using  either  a  Nominal  Group  Technique  or  Delphi  group  process.  The  researchers 
were  mostly  interested  in  evaluating  a  group’s  performance  with  respect  to  decision  quality  and 
decision  speed,  but  also  user  satisfaction.  As  a  whole,  the  results  from  many  such  laboratory  studies 
were  inconsistent. 

Notwithstanding  the  overt  differences  in  findings  from  the  laboratory  studies,  some  practi¬ 
tioners  in  the  GDSS  field  have  catapulted  the  technology  into  industry  and  are  now  (1991  to  the 
present)  reporting  significant  positive  findings  which  result  from  the  application  of  these  systems 
in  real  world  business  situations.  There  are,  however,  few  Phase  IV  GDSS  field  experiments  and 
studies  to  validate  or  substantiate  these  positive  findings.  The  lack  of  such  data  provides  a  unique 
opportunity  to  conduct  valuable  independent  research  applicable  to  Air  Force  activities. 

1.2  Research  Applicability  to  the  USAF 

The  potential  benefits  of  GDSS  use  in  real  organizational  settings  are  just  now  being  docu¬ 
mented.  As  GDSS  technology  matures  and  gains  widespread  acceptance,  Air  Force  personnel  could 
conceivably  use  these  systems  to  make  group  decisions  in  either  single  one- room  settings  or  between 
numerous  organizations  geographically  separated  around  the  world.  These  computer-supported 
group  sessions  may  better  enable  our  senior  leaders  to  make  informed,  yet  complex  decisions  in  a 
timely  manner  with  improved  coordination  of  USAF  personnel.  In  addition  to  improved  decision 
making,  GDSSs  may  alleviate  the  need  for  extensive  TDY  travel,  resulting  in  significant  cost  and 
time  savings.  The  underlying  problem  is  that  there  is  no  consensus  in  the  Information  Systems 
community  that  use  of  a  GDSS  improves  group  decision  making.  Likewise,  there  is  little  agreement 
that  users  will  be  satisfied  with  using  this  computer  technology,  nor  accept  it  for  widespread  use  in 
real  business  organizations.  Because  of  the  current  uncertainties  associated  with  employing  GDSS 
resources,  prospective  users  should  carefully  evaluate  the  potential  for  success  by  testing  GDSS 
capabilities  within  their  unique  environments. 

The  Air  Force’s  Human  Resources  Directorate  of  the  USAF  Armstrong  Laboratory  at  Wright 
Patterson  AFB,  Ohio  is  currently  establishing  a  GDSS  research  and  evaluation  facility  where  Air 
Force  groups  can  assemble  and  obtain  hands-on  experience  with  GDSS  technology.  A  primary 
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objective  of  the  USAF  GDSS  program  is  to  introduce  the  technology  to  many  different  organizations 
within  the  Air  Force  community  and  evaluate  not  only  GDSS  features  and  its  ability  to  enhance 
decision  making,  but  also  to  measure  user  perceptions  and  levels  of  acceptance.  Should  the  results 
from  these  studies  confirm  the  positive  results  proclaimed  by  commercial  proponents,  one  future 
outcome  of  the  Air  Force  GDSS  research  effort  may  be  to  operationalize  this  capability  by  infusing 
GDSS  into  the  workplace,  where  deemed  appropriate.  By  doing  so,  the  Air  Force  could  potentially 
benefit  from  improved  decision  making  which,  in  turn,  could  improve  organizational  productivity 
in  terms  of  reduced  cost  and  time  to  conduct  group  decision  processes.  Although  conventional 
meetings  will  continue  to  be  one  method  for  a  group  decision  making  process,  GDSS  technology 
may  prove  to  be  an  attractive  alternative  for  some  situations. 


1.3  Problem  Statement  and  Purpose  of  Research 

Currently,  there  is  limited  opportunity  for  GDSS  researchers,  in  academia  and  industry,  to 
examine  real  world  problems  in  a  group  decision  making  environment  while  simultaneously  applying 
appropriate  measures  of  experimental  control.  This  limitation  hinders  one’s  ability  to  observe  truly 
significant  results  in  the  GDSS  evaluation  process.  As  discussed  previously,  there  also  exists  a 
significant  difference  in  results  stemming  from  laboratory  experiments  (circa  1980-1991)  and  those 
of  the  more  recent  field  studies  (1991-1992).  Although  several  researchers  associated  with  the 
original  studies  have  proposed  explanations  for  the  inconsistent  results,  others  have  called  for 
additional  research  to  reconcile  the  differences.  For  example,  Ilze  Zigurs  in  “Methodological  and 
Measurement  Issues  in  Group  Support  Systems  Research”  states  that  “what  we  still  lack,  however, 
[are]  field  experiments  that  provide  semi- controlled  circumstances  [emphasis  added]  so 
we  can  tell  just  what  is  going  on  in  these  sites”  (44:122).  Zigurs’  proposal  to  bring  control  or 
experimental  structure  to  a  real  world  group  decision  making  environment  is  precisely  the  approach 
sought  in  this  research. 

The  purpose  of  this  study  was  evaluate  the  efficacy  of  GDSS  in  an  Air  Force  decision  making 
environment.  The  researcher  conducted  GDSS  experimentation  in  an  operational  Air  Force  situa¬ 
tion  with  sufficient  experimental  control  to  observe  statistically  meaningful  results.  Specifically,  the 
research  sought  to  objectively  evaluate  the  effectiveness  and  efficiency  of  GDSS-supported  groups 
compared  to  non-GDSS-supported  groups  in  a  choice  decision  making  process.  The  research  also 
attempted  to  compare  results  stemming  from  the  application  of  GDSS  computer  technology  to  the 
application  of  process  structure.  Finally,  the  research  sought  to  gain  an  understanding  of  the  user’s 
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satisfaction  with,  and  acceptance  of,  GDSS  technology  as  it  relates  to  the  group  decision  making 
process  and  its  outcomes. 


1.4  Research  Objectives 

There  are  several  research  objectives  which  support  the  purpose  of  this  study.  First,  the 
research  sought  to  measure  GDSS  and  process  structure  effects  as  they  related  to  performance 
outcomes  of  the  group  process:  1)  group  decision  making  efficiency  in  terms  of  decision  making 
speed  and  2)  group  decision  making  effectiveness  in  terms  of  consensus  reached.  Second,  the 
research  sought  to  measure  GDSS  and  process  structure  effects  in  relation  to  secondary  outcomes 
of  the  group  process:  1)  user  satisfaction  with  the  group  process  and  group  results;  and  2)  user 
perception  of  task  complexity,  group  conflict,  ability  to  remain  focused  on  the  task,  productivity  in 
terms  of  time  efficiency  and  level  of  consensus,  and  amount  of  process  structure.  These  objectives 
parallel  those  of  the  laboratory  and  field  studies  examined  in  the  GDSS  literature.  To  readily 
examine  these  objectives,  they  were  further  delineated  into  the  following  sub-objectives: 

1.  GDSS  Effects:  Performance  Outcomes 

(a)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
decision  speed. 

(b)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
consensus. 

2.  GDSS  Effects:  Secondary  Outcomes 

(a)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  satisfaction  with  the  decision  making  process. 

(b)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  satisfaction  with  the  results  of  the  decision  making  process. 

(c)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  perception  of  task  difficulty. 

(d)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  perception  of  the  group’s  ability  to  stay  task-focused. 

(e)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  perception  of  productivity  in  terms  of  time  efficiency. 

(f)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  perception  of  productivity  in  terms  of  ability  to  reach  consensus. 

(g)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  perception  of  the  group’s  level  of  conflict. 

(h)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  perception  of  the  level  of  process  structure. 

3.  Structure  Effects:  Performance  Outcomes 
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(a)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  decision  speed. 

(b)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  consensus. 

4.  Structure  Effects:  Secondary  Outcomes 

(a)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  satisfaction  with  the  decision  making  process. 

(b)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  satisfaction  with  the  results  of  the  decision  making  process. 

(c)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  perception  of  task  difficulty. 

(d)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  perception  of  the  group’s  ability  to  stay  task-focused. 

(e)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  perception  of  productivity  in  terms  of  time  efficiency. 

(f)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  perception  of  productivity  in  terms  of  ability  to  reach  consensus. 

(g)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  perception  of  the  group’s  level  of  conflict. 

(h)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  perception  of  the  level  of  process  structure. 


1.5  General  Approach 

The  Air  Force  Institute  of  Technology  (AFIT)  and  the  USAF  School  of  Aerospace  Medicine 
conducted  a  conference  at  Wright- Patterson  AFB,  Ohio  to  identify  critical  education  requirements 
for  non-environmental  professions  for  the  USAF  Environmental  Education  Master  Plan.  The  par¬ 
ticipants  of  this  conference  met  in  small  group  settings  and  mapped  applicable  training  levels  for 
51  different  environmental  tasks  to  each  of  approximately  30  USAF  professions.3  Each  member  of 
the  small  groups  voted  individually.  The  groups  conducted  a  cAoice-type  decision  making  process 
in  attempt  to  reach  consensus  using  a  five  point  scale  for  the  appropriate  training  levels.  For  ex¬ 
perimental  purposes,  participants  were  exposed  to  one  or  two  treatments  in  the  decision  making 
process:  facilitated  GDSS,  facilitated  manual,  and  unfacilitated  manual.  Facilitators  observed  and 
recorded  time  and  consensus  data  in  order  to  compare  decision  speed  and  consensus  levels  between 
experimental  conditions.  Post-process  questionnaires  were  administered  to  collect  subjective  user 
perceptions  and  levels  of  satisfaction.  The  data  was  analyzed  for  statistical  significance  in  support 
of  the  sub-objectives  identified  previously. 

3The  terms  “professions”  and  “career  fields”  are  used  interchangeably  in  this  document. 
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1.6  Scope  and  Limitations 


This  research  was  limited  to  evaluating  the  performance  of  small  groups  consisting  of  five 
participants.  These  groups  met  in  a  face-to-face  environment  and  performed  a  choice-type  decision 
making  process.  Real-world  conditions  constrained  the  experimental  design  and  the  actual  conduct 
of  the  research.  Further  explanation  of  these  limitations  is  provided  in  Chapter  III,  Methodology. 


1.7  Sequence  of  Presentation 

Chapter  II  provides  an  overview  of  previous  GDSS  research  with  particular  attention  to 
theoretical  principles,  dependent  variables  of  interest,  and  findings  of  experiments  which  compared 
GDSS  with  manual  (non-GDSS)  processes.  Chapter  III  continues  the  discussion  of  the  theoretical 
foundation  with  the  methodology  for  the  current  research.  Specifically,  this  chapter  describes 
the  equipment,  facilities,  people,  and  procedures  used  to  successfully  accomplish  this  field  study. 
This  chapter  also  provides  an  overview  of  the  statistical  tools  applied  in  the  analysis  process. 
Chapter  IV  provides  the  results  of  the  research,  and  finally  Chapter  V  presents  conclusions  and 
any  recommendations. 
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II.  Literature  Review 


2. 1  Introduction 

The  occasions  when  business  managers  or  military  leaders  make  “corporate-level”  decisions 
alone  are  rapidly  becoming  quite  rare.  Chief  executive  officers  and  military  commanders  alike 
surround  themselves  with  very  capable  and  intelligent  people  who  together  make  many  of  the  critical 
decisions  that  lead  to  the  betterment  of  their  overall  organization.  Over  the  past  two  decades, 
computer  technology  has  steadily  provided  a  means  to  improve  the  efficiency  of  decision  making 
by  providing  decision  makers  with  methods  for  quickly  consolidating  and  analyzing  information 
(e.g.,  using  database  and  spreadsheet  programs).  Likewise,  computer  support  has  led  to  increased 
effectiveness  in  managerial  decision  making:  improved  quality  of  the  decision  making  process, 
better  decisions,  and  more  positive  impacts  on  the  organization  (e.g.,  accurate  forecasts  of  future 
corporate  growth  using  operations  research  models).  Historically,  computers  have  been  used  at  the 
individual  decision  making  level.  Today,  the  potential  use  of  computers  to  facilitate  group  decision 
making — now  being  studied  in  academia  and  applied  in  industry — is  furthering  the  state-of-the-art 
in  information  technology. 

The  purpose  of  this  literature  review  is  to  acquaint  the  reader  with  several  concepts  and  issues 
pertaining  to  Group  Decision  Support  Systems  (GDSSs)  and  provide  an  overview  of  the  framework 
for  GDSS  research.  GDSS  is  a  relatively  new  concept.  Researchers,  educators,  and  writers  began 
publishing  GDSS-related  articles  in  the  early  1980s.  Significant  GDSS  research  efforts  have  been 
performed  primarily  at  major  universities,  such  as  the  University  of  Arizona,  the  University  of 
Minnesota,  Indiana  University,  and  others.  The  results  of  these  efforts,  however,  have  been  very 
mixed.  This  review  attempts  to  highlight  many  topics  concerning  the  study  of  GDSS  technology 
which  are  pertinent  to  the  current  research,  such  as: 

•  Some  activities  and  inherent  obstacles  associated  with  a  group  process. 

•  A  general  discussion  of  group  decision  support  systems  with  respect  to  decision  support 
systems. 

•  A  framework  for  GDSS  research  (settings,  components,  functions,  tasks,  variables  studied,  et 
cetera). 

•  An  overview  of  some  preliminary  results  of  both  laboratory  and  field  studies. 

2.2  Group  Decision  Making  Process 

A  brief  explanation  concerning  the  decision  making  process  and  the  types  of  activities  asso¬ 
ciated  with  this  process  is  useful.  Although  a  final  decision  is  often  made  by  a  single  individual — 
perhaps  a  top  executive  in  a  major  corporation — decision  making  is  usually  a  shared  process  with 
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members  at  various  hierarchical  levels  within  a  particular  organization.  Such  decisions  often  result 
from  collaboration  with,  and  cooperation  between,  members  of  a  particular  staff  in  a  meeting  where 
reaching  consensus  is  important. 

In  a  decision  making  process,  the  decision  maker(s)  must  identify  alternatives,  examine  all 
candidate  proposals  for  acceptability  and  risk,  evaluate  each  alternative  in  terms  of  a  set  of  objec¬ 
tives  or  requirements,  compare  and  rank  the  possible  outcomes,  and  finally  select  the  alternative 
which  offers  the  best  course  of  action  (5:30-31).  Gray  describes  these  types  of  group  meetings  by 
the  following  characteristics  (16:234): 

•  Meetings  are  a  joint  activity  engaging  5  to  20  people  all  of  equal  or  near-equal  status. 

•  The  activity  and  its  outputs  are  intellectual  in  nature. 

•  Groups  usually  retrieve,  generate,  and  share  information  in  attempt  to  reach  consensus  or 
make  a  decision. 

•  The  product  of  the  meeting  depends  largely  on  the  knowledge,  opinions,  and  judgments  of 
those  attending. 

•  Differences  in  opinion  are  settled  by  decree  of  the  ranking  person  or  by  negotiation  of  the 
participants. 

•  The  results  lead  to  subsequent  action  within  the  organization. 

One  benefit  of  group  decision  making  is  synergism — the  whole  being  greater  than  the  sum 
of  its  parts.  Each  participant  brings  a  unique  level  of  knowledge,  understanding,  and  experience 
to  a  group.  There  is  great  potential  for  taking  advantage  of  these  attributes,  but  this  potential 
may  not  be  fully  realized  due  to  underlying  destructive  group  dynamics,  including  I)  premature 
adjournment  due  to  lack  of  critical  information,  2)groupthink  (pressure  to  conform  to  consistent 
thinking  patterns  of  a  highly  cohesive  group),  3)  members  not  having  the  opportunity  to  speak 
or  unwilling  to  confront  issues,  4)  suppression  of  participation  due  to  monopolizing  personalities, 
and  5)  a  lack  of  focus  which  wastes  time  (3:112).  These  obstacles  certainly  influence  conventional 
group  meetings,  but  some  may  be  ameliorated  by  exploiting  several  reported  advantages  of  GDSSs 
(3:112): 

•  Groupthink.  GDSSs  can  provide  anonymity  of  individual  comments,  allowing  contradictory 
opinions  to  surface  without  fear  of  reprimand. 

•  Competition  for  Opportunity  to  Speak.  GDSSs  allow  simultaneous  contribution  of 
ideas/ comments. 

•  Monopolizing  Personalities.  GDSSs  allow  equitable  participation,  since  all  participants 
are  allowed  to  “voice”  opinions. 

•  Environments  Not  Conducive  to  Confrontation.  GDSSs  remove  personal  confrontation 
by  using  more  nonverbal  communication.  Ideas,  not  people,  become  the  focus  of  criticism. 

•  Lack  of  Focus.  GDSSs  enhance  task  focus  through  nonverbal  communications  and  group 
visual  provisions. 
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2.3  Delphi  Process  and  Nominal  Group  Technique  (23:73-74,  128) 


Two  popular  group  processes  which  have  influenced  GDSS  software  design  and  operational 
procedures  are  the  Delphi  process  developed  by  the  RAND  Corporation  and  the  Nominal  Group 
Technique.  These  processes  were  developed  in  an  attempt  to  improve  the  accuracy  of  group  decision 
making  (23:73). 

The  Delphi  group  process  is  primarily  used  in  intellective  problem  solving  tasks  where  a  right 
decision,  or  one  best  solution,  exists.  Members  work  toward  solving  usually  a  quantitative  problem, 
but  they  work  alone.  The  anonymous  decisions  of  fellow  group  members  are  provided  to  each  of  the 
individual  decision  makers.  Throughout  successive  rounds  of  this  decision  making  process,  there 
is  no  direct  interaction  between  group  members.  The  process  continues  until  the  group  reaches  a 
result  or  consensus  for  a  decision  (23:74). 

The  Nominal  Group  Technique  (NGT)  is  a  two-stage  group  process  used  primarily  for  creative 
and  planning  types  of  decisions.  In  an  initial  stage,  members  work  alone  to  individually  generate 
ideas  or  identify  alternatives  for  a  particular  problem.  During  the  second  stage,  the  members 
work  together  to  evaluate  alternatives  to  solve  a  particular  problem.  NGT  is  particularly  useful 
for  choice  decisions  (i.e.,  choosing)  where  voting  mechanisms  are  employed.  Individuals  publicly 
cast  an  initial  vote  without  prior  discussion.  This  process  is  contrasted  with  the  Delphi  process, 
since  here  an  initial  vote  is  publicly  cast,  but  is  followed  by  interaction  between  group  members. 
This  discussion  period  allows  individuals  to  explain  their  decision.,  before  a  group  decision  is  made 
(23:74,128). 

2.4  Discussion  of  Group  Decision  Support  Systems 

Before  fully  comprehending  the  concept  of  GDSS,  one  should  become  acquainted  with  De¬ 
cision  Support  Systems.  This  section  provides  information  highlighting  the  relationship  of  GDSS 
with  its  predecessor  Decision  Support  Systems,  and  it  identifies  several  goals  and  objectives  for 
GDSS  technology. 

2.4.I  GDSS  Relationship  to  Decision  Support  Systems  (DSSs).  Davis  states  that  “the  term 
DSS  generally  applies  to  [computer]  systems  that  are  designed  to  help  managers  evaluate  and 
analyze  complex  situations”  (5:xv).  Experts  in  the  DSS  community  qualify  this  characterization 
by  generally  defining  a  DSS  as  a  computer  system  designed  with  hardware  and  software  capabilities 
which  provide  real-time  (i.e.,  immediate)  analytical  support  to  assist  humans  in  making  effective 
decisions.  The  distinguishing  feature  that  sets  a  DSS  apart  from  other  computer  systems  is  its 
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ability  to  provide  real-time  “what-if’-type  analysis  for  a  user  (5:13).  It  is  important  to  note  that 
a  DSS  does  not  replace  the  decision  maker — he  or  she  still  makes  the  final  decision  (5:47). 

Since  a  GDSS  includes  components  of  a  DSS  (i.e.,  a  data  base,  report  generator,  user  interface, 
et  cetera),  a  GDSS  could  be  considered  a  DSS  if  only  one  person  used  the  system.  This  situation,  of 
course,  is  not  intended.  According  to  Gray,  the  requirements  for  a  GDSS  exceed  those  of  a  typical 
DSS  by  the  following  considerations  (16:237): 

•  Expanded  connectivity  with  a  communications  network  to  link  several  computers  together. 

•  Enhanced  software  to  provide  voting,  ranking,  and  other  group  decision  tools. 

•  Improved  availability  providing  additional  operational  uptime. 

•  Increased  setup  time  before  operational  use. 

•  Expanded  conference  facilities  with  elegant  furnishings. 

2-4-2  GDSS  Goals  and  Objectives.  GDSSs,  as  previously  stated,  support  more  than  a  single 
decision  maker  and  are  considered  a  subfield  of,  or  perhaps  an  evolution  in,  DSS  technology.  GDSSs 
offpr  the  potential  for  improved  facilitation  of  communications  and  decision  support  in  a  group 
decision  making  process.  Gray  discusses  one  motivation  for  using  GDSSs— access  to  automated  data 
retrieval,  manipulation,  and  presentation  in  a  meeting  environment.  Businesses  have  become  highly 
dependent  on  computer  support  in  the  work  place;  however,  when  meetings  are  convened,  managers 
and  decision  makers  leave  all  the  computer  support  behind.  These  computers  provide  essential 
access  to  corporate  data  and  applicable  software  models  upon  which  decisions  are  frequently  made. 
In  the  conference  room,  quite  often  the  only  available  information  technology  is  the  telephone  and 
the  overhead  projector  (16:234). 

Making  better  group  decisions  is  certainly  an  important  goal  for  GDSS  to  succeed.  Group 
participants,  however,  must  also  be  satisfied  with  the  process  and  the  results  produced  before  they 
will  be  willing  to  use  these  capabilities  extensively  over  the  long  term.  GDSS  experts  do  not  agree 
that  these  goals  can  be  simultaneously  achieved  (8:606). 

DeSanctis  and  Gallupe  have  provided  a  good  foundation  for  GDSS  research.  They  have  stated 
that  the  aim  of  GDSSs  is  to  “improve  the  process  of  group  decision  making  by  removing  common 
communication  barriers,  providing  techniques  for  structuring  decision  analysis,  and  systematically 
directing  the  pattern,  timing,  or  content  of  discussion”  (8:589).  They  also  said  that  the  objective 
of  GDSSs  is  to  “discover  and  present  groups  with  new  possibilities  and  approaches  for  making 
decisions"  (8:595). 
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2.5  Framework  of  the  GDSS  Research  Model 


GDSS  research  has  taken  many  forms;  however,  there  is  a  semblance  of  structure  within 
the  literature  which  is  brief y  discussed  in  this  document.  Specifically,  this  section  is  devoted  to 
addressing  characteristics  of  the  GDSS  research  in  terms  of  group  settings,  GDSS  components  and 
levels  of  features,  group  activities/functions,  decision  making  tasks,  and  the  variables  most  often 
measured  for  effectiveness. 


2.5.1  GDSS  Taxonomy  of  Group  Settings.  Researchers  are  evaluating  the  application  of 
GDSS  primarily  in  face-to-face  meetings;  however,  geographically  separated  (distributed)  meetings 
have  also  been  considered,  but  to  a  far  lesser  degree.  The  GDSS  configuration  varies  with  each 
meeting  situation.  Several  scenarios  are  possible,  depending  on  the  duration  of  the  decision  making 
session  (i.e.,  a  temporal  function)  and  the  physical  distance  (i.e.,  a  proximity  function)  between 
group  members  Although  neither  wholly  exhaustive  nor  mutually  exclusive,  the  following  general 
configurations  are  possible  (38:134),  (8:599-600): 

Decision  Room.  All  participants  meet  in  one  room  for  a  fixed  period  of  time.  Each  participant 
has  a  computer  terminal  to  perform  individual  work  and  to  transmit  information  to  a  common 
large  screen  display  for  public  viewing.  This  room  is  also  called  a  war  room  or  face-to-face 
conference. 

Legislative  Session.  Same  configuration  as  the  decision  room,  except  that  the  number  of  par¬ 
ticipants  has  increased  and  inputs  for  public  viewing  must  go  through  a  facilitator  (someone 
who  controls  the  GDSS  software  and  common  display  devices)  prior  to  being  displayed  on 
the  large  viewing  screen. 

Local  Area  Decision  Network.  Individuals  participate  from  terminals  in  their  own  work  areas. 
Meeting  members  could  be  dispersed  throughout  a  building  or  across  town.  This  configuration 
does  not  require  all  participants  to  remain  “on-line”  at  a  given  time  (temporal  function). 
An  electronic  facilitator  can  coordinate  participant  inputs  in  accordance  with  a  prearranged 
schedule. 

Teleconferencing.  Participating  individuals  are  geographically  distant  (proximity  function)  from 
one  another;  however,  all  are  located  in  respective  “local”  decision  rooms  interconnected  by 
audio,  and  perhaps  video,  communications  capabilities.  Conference  participants  meet  at  the 
same  time.  Teleconferencing  simply  supplements  the  existing  GDSS  capabilities. 

Remote  Decision  Making.  A  large  number  of  participants  are  widely  dispersed  and  may  par¬ 
ticipate  in  a  conference  similar  to  the  local  area  decision  network.  This  GDSS  configuration 
allows  either  same  time  or  different  time  use 


2.5.2  GDSS  Components  and  Decision  Room.  Turban  synthesized  the  typical  GDSS  archi¬ 
tecture  identified  by  DeSanctis  et  al  into  the  following  components  (38:132): 

1.  Hardware. 

(a)  Input/output  device,  preferably  a  microcomputer,  for  each  participant  to  perform  indi¬ 
vidual  work  or  to  send  information  forward  to  the  group. 
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(b)  Common  viewing  screen  to  display  information  to  the  entire  group. 

2.  Software  (Group  features  in  addition  to  individual  work  applications). 

(a)  Numerical  and  graphical  summarization  of  ideas  and  votes. 

(b)  Programs  for  calculation  of  weights  for  decision  alternatives,  anonymous  recording  of 
ideas,  formal  selection  of  a  group  leader,  and  elimination  of  redundant  inputs. 

(c)  Text  and  data  transmission  capabilities  among  participants,  between  members  and  the 
facilitator,  and  between  the  group  and  central  computer  processors. 

3.  People. 

(a)  Group  members. 

(b)  A  facilitator  or  chauffeur  who  operates  the  GDSS  hardware  and  software,  and  who 
displays  requested  information  on  the  common  large  screen. 

4.  Procedures. 

(a)  Rules  on  the  operation  of  the  GDSS  hardware  and  software. 

(b)  Rules  concerning  verbal  communications  among  members  and  the  flow  of  events  during 
the  meeting. 

As  noted  above,  GDSS  components  can  be  configured  in  a  variety  of  ways  within  the  GDSS 
taxonomy.  The  most  commonly  used  configuration  for  research  purposes  has  been  the  Decision 
Room.  One  of  the  first  such  facilities  was  the  PlexCenter  located  at  the  University  of  Arizona. 
This  facility  actually  has  two  decision  rooms;  the  first  and  smaller  facility  was  built  in  1985,  the 
second  and  larger  facility  was  completed  in  1987.  Figure  2.1  shows  a  schematic  of  the  first  decision 
room.  This  facility  is  equipped  with  16  networked  microcomputers  positioned  around  a  U-shaped 
conference  table.  Information  from  an  individual  workstation  or  aggregated  group  data  can  be 
transmitted  via  a  local  area  network  to  the  facilitator’s  workstation  and,  subsequently,  relayed  to 
the  large  projector  screen  for  public  viewing.  Several  breakout  rooms,  complete  with  networked 
microcomputers,  are  available  for  side  discussions.  The  second  facility,  shown  in  Figure  2.2,  has 
26  networked  microcomputers  and  up  to  60  people  can  use  the  room  simultaneously.  This  room 
has  two  large  display  screens  for  dual  projection  along  with  greatly  enhanced  audio  and  video 
capabilities  (42:19-20). 

Integral  to  this  facility  is  the  GDSS  software — originally  the  PLEXSYS  Planning  System.1 
The  PLEXSYS  Planning  System  is  actually  a  knowledge-based  software  system,  much  like  an 
expert  system,  with  an  inference  mechanism  which  accesses  both  explicit  and  implicit  knowledge 
bases  (29:836).  At  some  point  in  the  1980s,  the  University  of  Arizona,  along  with  12  other  schools, 
was  awarded  a  $2  million  grant  from  International  Business  Machines,  Inc  (IBM)  which  was  used 

'The  author  assumes  that  the  University  of  Arizona  currently  uses  Ventana’s  Group  Systems  software  developed 
in  cooperation  with  IBM  as  an  evolution  to  PLEXSYS. 
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Figure  2.1.  PLEXCENTER  Decision  Room  #1  (42:20). 
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to  expand  existing  GDSS  hardware  and  software  capabilities  (30:185).  This  corporate- academic 
relationship  matured,  and  today  IBM  and  the  University  of  Arizona  together  have  built  more 
than  20  GDSS  facilities  (called  Decision  Support  Centers-DSCs)  with  approximately  30  additional 
facilities  planned  worldwide.  The  first  IBM  facility  was  built  in  1987  at  an  IBM  manufacturing 
plant  in  New  York  (30:185-186).  Much  of  the  field  study  data  (700-plus  GSS  sessions)  published  by 
the  University  of  Arizona  and  IBM  have  been  collected  at  this  facility.  Although  the  specific  details 
are  not  well-documented,  Dr  Jay  Nunamaker,  Head  of  the  University  of  Arizona  MIS  department, 
has  developed  a  new  generation  GDSS  software  package  called  GroupSystems  which  is  marketed 
by  Ventana  Corporation.  A  similar  GDSS  package,  developed  in  cooperation  with  IBM  and  called 
Team  Focus ,  is  being  used  at  IBM’s  DSCs. 

In  addition  to  GDSS  hardware,  software,  and  facilities,  the  role  of  the  facilitator  is  critically 
important,  and  should  not  be  underestimated.  Although  rarely  included  in  most  of  the  laboratory 
studies,  the  facilitator  has  an  essential  factor  in  the  use  of  GDSS  in  field  studies.  The  facilitator 
performs  a  variety  of  functions  which  include  (31:134): 

•  Guiding  the  group  through  technical  aspects  of  the  GDSS  process. 

•  Chairing  meetings,  setting  agendas,  and  assessing  the  need  to  adjust  meeting  schedules. 

•  Taking  necessary  actions  to  improve  group  interaction. 

•  Assisting  in  planning  meeting  agendas  with  group  leaders. 

2.5.3  Activities  or  Functions  Associated  with  GDSSs.  Meeting  activities  using  GDSS  tech¬ 
nology  are  similar  to  those  of  conventional  meetings,  except  that  more  structure  is  added  to  the 
process  along  with  a  wealth  of  electronic  assistance.  Many  GDSSs  today  include  software  tools 
for  idea  generation  (i.e.,  electronic  brainstorming),  topic  evaluation,  ranking  and  voting,  et  cetera. 
These  tools  are  designed  to  allow  groups  to  address  a  relatively  unstructured  problem,  derive  al¬ 
ternative  solutions,  and  ultimately  make  a  final  decision  which  they  will  later  implement.  Consider 
the  primary  software  tools2  offered  by  GroupSystems i3  (40:2-2,  2-4): 

Electronic  Brainstorming.  An  unstructured  idea  generation  tool  allowing  participants  to  share 
ideas  on  a  question  or  issue. 

Idea  Organization.  A  feature  used  to  categorize  comments  and  generate  ideas,  such  as  those 
created  during  brainstorming. 

Topic  Commenter.  An  idea  generator  which  invokes  a  more  detailed  structure.  Uses  a  predefined 
list  of  topics  which  participants  comment  on. 


2This  is  a  partial  listing  of  the  more  popular  software  tools. 

3  These  tools  represent  the  next  generation  of  the  original  PLEXSYS  Planning  System. 
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Alternative  Evaluation.  A  method  which  allows  participants  to  rate  a  list  of  alternatives  against 
an  established  criterion. 

Vote.  The  tool  which  allows  participants  to  vote  on  a  particular  topic  or  issue.  Results  can  be 
statistically  analyzed  and  shown  to  the  group. 

Policy  Formulation.  A  text  editing  tool  which  allows  groups  to  iteratively  comment  on  a  pro¬ 
posed  policy  or  mission  statement  until  the  group  reaches  consensus. 

Group  Matrix.  A  tool  used  to  establish  relationships  between  entities  in  a  2  x  2  matrix,  either 
textually  (from  a  predetermined  list  of  words)  or  numerically.  Cells  within  the  matrix  indicate 
the  level  of  consensus. 

Stakeholder  Identification.  A  method  to  identify  participants  who  either  impact  or  are  im¬ 
pacted  by  a  proposed  plan  of  action. 

2.5.4  GDSS  Typology  of  Tasks.  In  the  early  research,  there  was  a  need  to  standardize  the 
classification  of  different  group  tasks  to  focus  the  experimentation  effort.  The  study  of  group 
tasks  is  not  new.  In  fact,  such  studies  stem  back  to  the  late  1800s;  consequently,  the  foundation 
for  categorizing  group  tasks  has  already  been  laid.  GDSS  researchers  have  consistently  applied 
the  typology  of  tasks  formulated  by  Joseph  McGrath,4  particularly  in  research  conducted  at  the 
University  of  Minnesota  and  the  University  of  Arizona  (44:116).  McGrath  stated  that  “if  we  want 
to  learn  about  groups  as  vehicles  for  performing  tasks,  we  must  either  (a)  assume  that  all  tasks  are 
alike... or  (b)  take  into  account  differences  in  group  performance  as  they  arise  from  differences  in 
tasks”  (23:53). 

McGrath’s  Group  Task  Circumplex  (reference  Figure  2.3)  provides  the  needed  framework 
to  categorize  and  describe  tasks  associated  with  GDSS  experiments  and  allows  the  comparison 
of  similar  studies.  McGrath  categorized  group  tasks  into  a  classification  schema  which  he  stated 
should  be  “mutually  exclusive  (a  task  fits  in  one  and  only  one  category),  collectively  exhaustive 
(all  tasks  fit  into  a  category),  and  logically  related  to  one  another”  (23:60).  The  model  is  divided 
into  four  quadrants  of  activities  or  processes:  Generate,  Choose,  Negotiate,  and  Execute;  however, 
only  the  first  three  directly  relate  to  GDSS  group  processes.  Each  of  the  Group  Task  Circumplex 
model’s  quadrants  are  subdivided  into  specific  tasks — the  level  at  which  comparisons  are  easily 
made.  These  specific  tasks  are  best  understood  in  summary  form  and  are  shown  in  Figure  2.4. 

Having  a  common  model  for  the  typology  of  tasks,  researchers  also  needed  to  classify  their 
experiments  according  to  a  taxonomy  of  GDSS  group  settings.  Figure  2.5  shows  two  factors  which 
captured  the  essence  of  this  taxonomy:  Group  Size  and  Member  Proximity.  This  taxonomy  does  not 
provide  a  third  dimension  for  the  temporal  function  (i.e.,  same  or  different  times).5  Combining  the 

4 Based  on  the  previous  works  of  Shaw,  Carter,  Hackman,  Steiner,  Shiflett,  Taylor,  Lorge,  Davis,  and  Laughlin 
(23:60). 

5This  void  is  probably  due  to  the  fact  that  the  research  appears  to  be  limited  temporally  to  “same  time”  settings. 
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Figure  2.3.  Group  Task  Circumplex — Typology  of  Tasks  (23:61). 


QUADRANT  I:  GENERATE 

Typel.  Planning  Tasks:  generating  plans. 

Type  2.  Creativity  Tasks:  generating  ideas  and  brainstorming. 

QUADRANT  II:  CHOOSE 

Type  3.  Intellective  Tasks:  solving  problems  with  correct  answers. 

Type  4.  Decision-Making  Tasks:  dealing  wSh  tasks  for  which  the  preferred 
or  agreed  upon  answer  is  the  correct  one. 

QUADRANT  III:  NEGOTIATE 

Type  5.  Cognitive  Conflict:  resolving  conflicts  of  viewpoint,  not  interests. 

Type  6.  Mixed-Motive  Tasks:  resolving  conflicts  of  motive-interest,  namely 
negotiations  and  bargaining. 

QUADRANT  IV:  EXECUTE 

Type  7.  Contests/Battles:  resolving  conficts  of  power,  competing  for  victory 
(e.g.,  wars,  competitive  sports). 

Types.  Performances:  peychomotor  tasks  performed  against  objective  or 
absolute  standards  of  excelence  (e.g ,  physical  tasks,  some  sports). 

Figure  2.4.  Key  Task  Concepts  for  McGrath’s  Group  Task  Circumplex  (23:62). 
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taxonomy  of  GDSS  settings  with  the  McGrath’s  Task  Circumplex  led  to  the  contingency  perspective 
for  GDSS  research  recommended  by  DeSanctis  and  Gallupe  (reference  Figure  2.6).  They  referred 
to  group  size,  proximity,  and  task  type  as  “environmental  contingencies  critical  to  GDSS  design” 
(8:589). 

2.5.5  GDSSs  Support  to  Decision  Making  Tasks.  Already  introduced,  group  dynamics  can 
be  obstacles  which  impede  progress  or  paralyze  communications  among  group  participants.  For 
example,  one  or  two  members  may  dominate  the  entire  discussion.  Timid  or  soft-spoken  people 
may  be  unwilling  to  contribute  their  ideas  to  a  discussion,  especially  if  the  issues  are  considered 
sensitive.  On  the  contrary,  some  people  have  too  much  to  offer,  especially  bosses  who  feel  they 
must  comment  on  every  point  brought  up  in  a  particular  discussion  (10:26).  Other  problems  may 
include  “extreme  influence  of  high-status  members  and  lack  of  acknowledgment  of  the  ideas  of  low 
status  members  or  the  low  tolerance  of  minority  or  controversial  opinions”  (8:596-597). 

Destructive  group  dynamics  hinder  the  effectiveness  of  group  decision  making.  Because 
GDSSs  allow  simultaneous  contribution  of  inputs  by  all  members,  destructive  group  dynamics 
should  not  be  as  significant.  Another  significant  feature  of  a  GDSS  is  the  ability  for  individuals  to 
anonymously  make  comments  or  suggestions.  Nonverbally,  they  input  information  through  their 
personal  computer  for  all  to  view  on  a  common  display  device  (i.e.,  a  large  screen  projection).  This 
feature  frees  the  individual  from  personal  inhibitions  to  make  open  and  honest  comments  about  a 
particular  subject.  As  Finley  states,  the  system  “forces  attendees  to  tell  the  truth”  (10:26).  The 
group  and  the  boss  are  forced  to  consider  the  merits  of  all  inputs  in  an  unbiased  manner.  In  other 
words,  all  inputs  receive  equal  consideration  without  regard  to  the  status  or  credibility  of  the  indi¬ 
vidual  who  thought  of  the  idea.  Johnson  points  out  that  anonymity  may  not  be  such  a  good  idea, 
however.  For  instance,  she  states  that  anonymity  “shields  people  who  have  not  thought  through 
their  views  carefully,”  and  it  can  impede  the  verbal  exchange  that  is  necessary  for  clarification  and 
feedback  of  new  or  confusing  ideas  (19:124).  Generally  speaking,  the  literature  appears  to  support 
a  positive  position  that  anonymity  potentially  leads  to  improved  decision  quality. 

Another  GDSS  attribute  is  the  ability  to  keep  meetings  on  track  with  participants  focused 
on  information  provided  visually  on  the  large  viewing  screen.  The  desired  end  result  is  increased 
productivity  and  efficiency  measured  in  terms  of  decision  making  speed.  Decision  making  speed, 
however,  can  be  controversial.  Some  experts  feel  that  speed  in  reaching  a  particular  decision  is 
not  as  important  a  measure  as  the  actual  quality  of  the  decision.  In  fact,  by  considering  a  larger 
number  of  alternatives  to  a  problem — perhaps  due  to  increased  participation — decision  time  may 
actually  increase,  but  the  end  product  may  be  a  much  improved  final  decision.  Essentially,  the  use 
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Figure  2.5.  GDSS  Taxonomy  of  Group  Settings  (8:598). 


Figure  2.6.  Contingency  Perspective  for  GDSS  Research  (8:591). 
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Table  2.1.  GDSS  Features  to  Support  Group  Tasks  (8:601). 


TASK  PURPOSE 

TASK  TYPE 

GDSS  LEVEL 

POSSIBLE  SUPPORT  FEATURES 

Planning 

1 

Large  Screen  Display,  Graphical  Aide 

3 

Planning  Toole  (e.g.,PERT) 

Creativity 

1 

Anonymous  lapat  of  Ideas. 

Pooling  It  Display  of  Ideas 

a 

Nominal  Gronp  Techniqne, 

Brainstorming 

Choose 

Intellective 

i 

Data  Access  It  Display, 

Synthesis  it  Display  of  Rationales 

a 

Aids  to  Finding  Correct  Choice, 

Forecasting  Models, 

Multiattribute  Utility  Models 

a 

Rale-Based  Discassion 

Preference 

i 

Weighing  it  Ranhiag, 

Voting  Schemes 

a 

Social  Jndgment  Models, 

Aatomated  Delphi 

s 

Rale- Based  Discassion  with 

Bqaal  time  to  Present  Opinion 

Negotiate 

Cognitive  Conflict 

l 

Sammary  1 1  Display  of  Opinions 

a 

Social  Jndgment  Analysis 

a 

Aatomatic  Mediation, 

Automate  Robert’s  Rales 

Mixed  Motive 

i 

Voting  Solicitation  it  Sammary 

a 

Stakeholder  Analysis 

a 

Rale  Base  for  Controlling 

Opinion  Expression, 

Automate  Parliamentary  Procedure 

of  GDSS  alters  the  nature  of  participation  and  the  communication  process  of  the  group,  hopefully 
resulting  in  better  decision  quality.  The  greater  the  impact  to  communication,  the  “greater  the 
potential  for  an  improved  decision  making  process”  (8:590  591). 

2.5.6  Taxonomy  of  GDSS  Levels.  Based  on  McGrath’s  work,  DeSanctis  and  Gallupe  recom¬ 
mended  possible  GDSS  features  for  three  levels  of  GDSS  sophistication.  These  features  are  shown 
in  Table  3.1.  To  date,  the  research  conducted  and  the  commercially  available  GDSS  software  adhere 
primarily  to  Level  1  capabilities. 


2.5.7  Group  Decision  Support  Variables.  There  are  numerous  factors  which  influence  any 
group  process,  many  of  which  have  been  examined  in  the  laboratory  and  field  studies.  Figure  2.7 
shows  the  model  Gray6  proposed  that  contained  the  important  categories  of  the  different  variables 

6  Gray  summarized  data  taken  from  an  unpublished  working  paper  of  DeSanctis  and  Gallupe  entitled  “Information 
System  Support  for  Group  Decision  Making,”  University  of  Minnesota  (undated). 
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affecting  group  decision  making.  These  variables  fall  beneath  three  major  areas:  inputs,  process, 
and  outputs.  Of  these  areas,  the  actual  categories  of  variables  are  as  follows  (16:239): 

1.  Inputs:  Group  Factors,  Individual  Factors,  Contextual  Factors,  and  Technological  Factors. 

2.  Process:  Information  Processes,  Decision  Techniques  Employed,  Group  Dynamics  Aspects. 

3.  Outputs:  Performance  and  Secondary  Outcomes. 

Discussion  of  the  applicable  variables  in  an  experiment  is  important  when  conducting  GDSS 
research.  One  should  identify  which  variables  included  in  the  study  are  controlled,  measured,  or  not 
considered.  This  identification  helps  to  precisely  characterize  the  study  so  that  others  may  conduct 
comparative  analysis  with  similar  experiments  or  replicate  the  study,  if  so  desired.  Gray  points  out 
that  “the  ability  to  replicate  is  important  for  validating  existing  theories.”  (16:239).  One  specific 
theory  is  that  the  use  of  GDSS  improves  performance  outcomes,  such  as  decision  speed  or  decision 
quality.  This  leads  to  the  discussion  of  measures  of  effectiveness. 


INPUTS 
Group  Factors 

Croup  History 
Gro*q>  Cohesivenest 
Group  sis* 

Facilitator's  Role 
Leadership  Type 

Individual  Factors 

hi  ant*  r  Skills 
Attitudes 

Personality  Traits 
Expectations 
Perceived  Status 

Contextual  Factors 

Task  Characteristics 
Member  proximity 
Org'l  Award  Structure 
Time  Pressure 
GDSS  Traimng/Si^port 

Technical  Factors 

HardmrtbSoftwart  Features 
TelecemmtmcatUms  Technology 
Communications  Media 
System  Configuration 


PROCESS 


Information  Process 

Information  Retrieval 
//formation  Sharing 
Information  Use 


Type  of  Dedaion  Technique 

Ordinary  Process 
Nominal  Grot?  Teckmque 
Delphi 

Bmuutonmng 
Social  Judgement  Analysis 
Statistical  Aggregation 


Group  Dynamics 
Norm  Development 
Partidpation 

Power  Relationships 
Corfbct 


OUTPUTS 


Performance  Outcomes 

Decision  Speed 
Decision  Quality 


Secondary  Outcomes 
Extent  of  GDSS  Use 
Attitudes 

Level  ef  Consensus 
Decision  Cotfidence 
Sa&jaction  with  GDSS 
Number  of  Meetings 


Figure  2.7.  Group  Decision  Support  Variables  (16:239). 


2.5.8  Measures  of  Effectiveness.  In  any  experiment,  results  must  be  evaluated  against  cri¬ 
teria  in  order  to  measure  the  degree  of  success  or  failure.  The  results  which  are  evaluated  are  the 
outcome  variables  proposed  by  Gray.  These  variables,  identified  as  the  dependent  variables,  fall 
into  two  primary  areas  (16:239): 


2-14 


1.  Performance  Outcomes. 

(a)  Decision  Speed. 

(b)  Decision  Quality. 

2.  Secondary  Outcomes. 

(a)  Extent  of  GDSS  Use. 

(b)  Participant  Attitudes. 

(c)  Level  of  Consensus. 

(d)  Decision  Confidence. 

(e)  Satisfaction  with  GDSS. 

(f)  Number  of  Group  Meetings  to  Reach  a  Decision. 

Performance  and  secondary  outcomes  have  historically  been  the  focus  for  numerous  experi¬ 
ments;  however,  three  particular  variables  stand  out  as  being  the  most  often  evaluated:  decision 
quality,  level  of  participation,  and  satisfaction  with  the  process  (7:600).  These  variables  are  derived 
from  both  the  output  and  process  categories.  Zigurs  noted  that  the  task-related  variables  most  of¬ 
ten  studied  were  decision  speed,  decision  quality,  consensus,  and  thoroughness  of  analysis.  He  also 
noted  that  the  satisfaction  variables  most  often  studied  were  decision  confidence,  satisfaction  with 
the  process,  and  satisfaction  with  the  decision  (44:117).  DeSanctis  and  Gallupe  state  that  a  dual 
purpose  to  facilitate  high  quality  decisions  and  a  high  sense  of  user  satisfaction  using  GDSS  may 
be  contradictory.  They  submit  that  researchers  should  be  concerned  with  both  aspects;  however, 
improving  the  efficiency  and  effectiveness  of  group  decision  making  are  the  more  important  criteria 
(8:606). 

Explaining  the  output  variables  further,  DeSanctis  and  Gallupe  state  that  the  “outcomes 
of  decision  meetings  may  be  measured  by  the  quality  and  timeliness  of  the  decision,  satisfaction 
with  the  decision,  cost  or  ease  of  implementation,  member  commitment  to  implementation,  and 
the  group’s  willingness  to  work  together  in  the  future”  (8:592).  To  date,  GDSS  laboratory  and 
field  studies  comparing  GDSS  groups  to  manual  groups,  for  instance,  have  not  resulted  in  similar 
conclusions  regarding  these  variables.  In  fact,  the  results  have  been  overtly  contradictory. 

2.5.9  Conflicting  Empirical  Results.  Researchers  have  drawn  a  number  of  contradictory 
conclusions  concerning  a  GDSS’s  ability  to  improve  the  group  decision  making  process.  Compar¬ 
ison  of  the  results  taken  from  numerous  laboratory  studies,  predominantly  using  undergraduate 
students  as  participants,  has  been  largely  inconclusive.  In  1988  Dennis  et  al  captured  many  of  the 
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Table  2.2.  Experimental  Research:  GDSS  versus  Non-GDSS  (7:600). 


STUDY 

DECISION 

QUALITY 

CONSENSUS 

DECISION 

TIME 

SATISFACTION 

WITH  PROCESS 

SATISFACTION 

WITH  OUTCOME 

St*eb  tc 

Johnson,  1981 

Better 

Increased 

Increased 

Increased 

Lewis,  1982 

Better 

No  Effect 

Ruble,  1964 

No  Effect 

Gallupe  et  *1,  1988 

Better 

Increased 

Decreased 

Decreased 

BcaacUir,  1967 

No  Effect 

No  Effect 

No  Effect 

Watson  et  *1,  1986 

Wortt 

No  Effect 

Decreased 

Zigars,  1987 

Better 

A.  Euton,  1986 

No  Effect 

No  Effect 

No  Effect 

Increased 

G-  Easton,  1986 

No  Effect 

Lett  Likely 

Faster  in  Face-to-Face 

No  Effect 

Jarvenpaa  el  al,  1968 

Better 

No  Effect 

inconsistencies  that  existed  (reference  Table  2.27)  in  those  studies  which  compared  groups  using 
GDSS  (decision  rooms)  to  non-GDSS  groups.  The  contradictions  were  obvious. 

2.5.10  Accounting  for  the  Differences.  Dennis  et  al  stated  that  the  inconsistent  results  were 
derived  from  studies  which  used  different  GDSSs:  different  facility  design  philosophies,  different 
software,  and  different  measures  of  the  dependent  variables.  They  said  that  “there  is  so  much 
variation  across  these  studies  that  generalizations  become  problematic  and  cannot  be  made  reliably” 
(7:602).  In  essence,  Dennis  et  al  claimed  that  the  vast  differences  in  the  laboratory  research 
methods  made  these  experiments  incomparable.  Pinsonneault  and  Kraemer  disagreed.  In  1990, 
they  separated  the  variables  from  these  and  other  GDSS  experiments  into  four  broad  categories 
for  comparison:  contextual,  independent,  intervening,  and  dependent.  They  made  a  distinction 
between  Group  Decision  Support  Systems  and  what  they  called  Group  Communication  Support 
Systems  (GCSSs)  features.  They  concluded  that  there  were  favorable  comparisons  with  relatively 
consistent  findings  (32:146-152). 

Although  not  mutually  exclusive,  GDSSs  contained  experiments  which  primarily  added  struc¬ 
ture  to  the  group  decision  process  (i.e.,  brainstorming,  ranking,  voting);  whereas,  GCSSs  were  ex¬ 
periments  which  merely  provided  information  aids  to  support  communications  (i.e.,  electronic  mail, 
teleconferencing,  local  area  networks).  Pinsonneault  and  Kraemer  concluded  that  GDSSs  had  more 
positive  impacts  on  group  processes  than  did  GCSSs  as  shown  in  Table  2.3  (32:143-159). 

Two  aspects  of  the  previous  discussion  are  very  interesting.  First,  it  is  not  totally  clear  how 
Pinsonneault  and  Kraemer  assigned  the  experiments  into  the  GDSS  and  GCSS  classes  once  the 
variables  were  identified.  Second,  six  of  the  ten  “GDSS”  experiments  which  they  called  comparable 
originated  from  the  University  of  Arizona.  It  was  here  that  Dennis  et  al  had  conducted  research 

'  This  table  does  not  contain  all  the  variables  Dennis  et  al  identified,  but  only  those  most  pertinent  to  the  current 
research  effort. 
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Table  2.3.  Comparison  of  GDSS  to  GCSS  (32:151,  154). 


DECISION 

DECISION 

CONFIDENCE 

GROUP 

TIME 

QUALITY 

IN  DECISION 

SATISFACTION 

CONSENSUS 

GDSS 

Negative 

Positive 

Positive 

Positive 

Positive 

GCSS 

Potitive 

Positive 

Negative 

r 

? 

and  stated  that  most  of  the  experiments  were  incomparable.  Gray  et  al  made  several  interesting 
observations  as  well. 

Gray,  Vogel,  and  Beauclair  developed  an  alternate  method  for  determining  which  experiments 
were  similar  and,  therefore,  could  be  appropriately  compared.  They  dissected  the  Pinsonneault- 
Kraemer  classification  method8  into  a  detailed  level  of  20  variables  clustered  into  six  metavariables. 
Some  of  these  20  variables  were,  subsequently,  broken  into  indicators  which  could  be  scored  with 
predetermined  numerical  values.  Finally,  weights  were  applied  in  order  to  differentiate  the  relative 
importance  of  each  indicator  (15:162-166).  The  example  in  Figure  2.8  illustrates  the  approach. 

Once  all  the  numerical  assignments  were  completed,  all  experiments  were  evaluated  to  deter¬ 
mine  which  had  nearly  equal  values  for  all  variables  using  Equation  2.1  (15:169).  This  equation 
provided  the  absolute  difference  between  all  experiments,  evaluated  pairwise.  If  the  difference  be¬ 
tween  a  pair  of  experiments  was  five  or  less,  they  were  deemed  close  (i.e.,  comparable).  Gray  et 
al  plotted  these  experiments  on  a  two-dimensional  graph  to  illustrate  similar  experiments  (15:168- 
170). 


*=i 


(2.1) 


where  v(k,  i)  and  v(k,j)  represent  the  variable  k  in  experiments  »  and  j,  respectively,  and  m  is  the 
number  of  variables  used  in  the  comparison. 

Gray  et  al  commented  that  “we  have  taken  11  of  the  12  experiments  that  are  considered  here 
and  also  in  Pinsonneault  and  Kraemer  (1990). ..[the  results]  show  no  clustering  of  experimental 
conditions  within  what  Pinsonneault  and  Kraemer  call  GDSS  studies.. .or  those  they  call  GCSS” 
(15:173-174).  To  make  another  counter  argument,  the  researcher  noted  that  half  of  the  so-called 
“conflicting”  experiments  considered  by  Dennis  et  al  to  be  incomparable  appeared  as  being  similar 
using  Gray’s  analysis.  This  author  concludes  that  there  is  no  resolution  for  the  conflicting  results 
of  the  GDSS  laboratory  studies.  Vogel  and  Nunamaker  state: 


8The  Pinsonneault- Kraemer  classification  of  variables  is  strikingly  similar  to  that  documented  by  Gray  (1987) 
based  on  the  unpublished  work  of  DeSanctis  and  Gallupe — the  true  origin  is  certainly  unclear. 
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...Many  different  results  occur  due,  in  part,  to  differences  in  technology,  task,  group 
size,  leadership,  and  other  potentially  interacting  variables.  A  caveat  also  exists  in 
terms  of  degree  of  experimentation  rigor,  measurement  sophistication,  and  accountabil¬ 
ity  for  confounding  effects... we  are  still  a  long  way,  however,  from  understanding  the 
implications  of  GDSS  on  group  process  and  outcomes  (42:19). 

Dennis  et  al  examined  24  variables  on  which  the  previous  laboratory  studies  varied  and 
concluded  that  the  results  were  not  contradictory — they  were  simply  not  comparable,  since  the 
studies  themselves  were  not  comparable  (44:122).  Zigurs  responded  to  this  statement  by  saying 
that  “what  we  still  lack,  however,  [are]  field  experiments  that  provide  semicontrolled  circumstances 
so  we  can  tell  just  what  is  going  on  in  these  sites  (44:122). 

Rao  et  al  state  that  these  inconsistencies  result  from  the  researchers’  failure  to  develop  hy¬ 
potheses  and  interpret  results  based  on  theoretical  models.  Research  is  moving  forward  without 
the  reconciliation  of  these  inconsistent  findings.  Rao  states  that  researchers  need  to  link  GDSS 
support  features  to  theoretical  models  such  as  communications,  minority  influence,  and  limitations 
of  human  information  processing  and  computational  abilities  (33:1347-1351).  Turban  also  notes 
the  disparities  which  exist  between  laboratory  experiments  and  field  studies.  He  advocates  that  re¬ 
searchers  embark  on  studies  in  both  arenas  and  apply  the  conclusions  drawn  from  experimentation 
to  use  of  GDSSs  in  business  organizations  (38:140). 

Vogel  et  al  have  conducted  numerous  studies  within  academia  and  business.  They  contend 
that  GDSSs  can  be  successful  in  meeting  the  goals  of  effectiveness,  efficiency,  and  user  satisfaction. 
They  state,  however,  that  “failure  to  capture  and  implement  ”  the  following  items  can  “easily  result 
in  adverse  effects”  (41:126-127): 

•  Facilities  must  provide  a  professional  setting  using  sophisticated  software  and  hardware  which 
are  well-organized  and  effectively  supported. 

•  There  must  be  the  ability  to  accommodate  groups  of  sufficient  size  that  may  vary  in  compo¬ 
sition  and  experience  which  address  real  and  complex  tasks. 

•  Facilitation  must  be  technically  competent  and  appreciate  group  dynamics  as  well  as  a  re¬ 
search  orientation  that  encompasses  a  multi-disciplinary  approach. 

2.5.11  Progress  Made  in  Industry.  There  is  a  virtual  void  of  GDSS  field  experiments  and 
field  studies,  except  those  associated  with  Dr  Jay  Nunamaker,  Head  of  the  Department  of  Manage¬ 
ment  Information  Systems  at  the  University  of  Arizona.  Using  Ventana  Corporation’s  GroupSys- 
tems,  his  research  has  largely  focused  on  the  commercial  sector,  primarily  International  Business 
Machines,  Inc  (IBM).  Nunamaker  et  al9  proclaim  overwhelming  success  in  applying  GDSS  tech- 

9  Vogel,  Dennis,  and  others  are  progeny  of  the  University  of  Arizona  GDSS/GSS  resea.  ch  efforts — either  as  doctoral 
students  or  faculty. 
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oology  in  the  held — directly  contradicting  the  results  of  the  earlier  laboratory  studies.  Vogel  et  al 
summarize  some  of  these  results  (42:24): 


The  efficiency  and  effectiveness  of  these  methods  have  proved  to  be  overwhelmingly 
positive.  Project  calendar  days  have  been  reduced  by  orders  of  magnitude.  The  number 
of  meetings  have  been  reduced  accordingly.  Person-hours  expended  have  been  dramat¬ 
ically  reduced,  with  an  average  savings  of  55%  based  on  experience  with  comparable 
unsupported  groups.  Comments  have  praised  the  fairness  and  comprehensiveness  of  the 
process  and  a  desire  to  use  the  facility  in  the  future.  Satisfaction  measures  have  been 
especially  positive.  Group  members  consistently  felt  that  the  computer-aided  process  is 
better  than  the  manual  one  in  terms  of  ideas  generated,  goal  achievement,  commitment 
generation,  fairness,  and  efficiency.  The  facility  has  never  been  advertised,  yet  is  now 
fully  booked  with  groups  based  on  word-of-mouth  of  successful  use. 

While  GDSS  researchers  continue  to  grapple  with  the  inconsistent  results  from  the  laboratory 
studies,  businesses  are  beginning  to  realize  actual  productivity  gains  (i.e.,  real  dollar  savings) 
using  GDSS.  Commercially  available  GSS  software  packages,  such  as  OptionFinder,  GroupSystems, 
TeamFocus,  VistonQuesi,  and  others  are  currently  being  used  in  industry  (28:3). 10  Kirkpatrick 
points  out  that  the  potential  for  GDSS  paybacks  is  tremendous,  considering  the  average  manager 
spends  almost  half  the  work  day  in  meetings.  Kirkpatrick  provides  some  fairly  impressive  results 
and  substantial  corporate  commitments  to  using  GDSS  technologies  (20:93-96): 

•  Boeing  cut  the  time  needed  to  complete  a  wide  range  of  team  projects  by  91%  and  saved  on 
average  $6,700  per  meeting  due  to  reduced  employee  time. 

•  IBM  reported  project  planning  time  reductions  of  56%. 

•  J.P.  Morgan,  Price  Waterhouse,  Marriott,  and  other  companies  started  using  GDSS  and/or 
have  built  a  Decision  Room  for  GDSS  purposes. 

•  IBM  is  in  the  process  of  building  50  Decision  Rooms  worldwide  which  it  plans  to  rent  to  other 
companies  at  $2,000  to  $5,000  per  day. 

Business  productivity  gains  have  been  significant.  For  example,  Boeing,  using  the  GDSS  soft¬ 
ware  package  TeamFocus,  cut  the  time  to  design  a  standardized  control  system  for  complex  machine 
tools  from  one  year  to  35  days.  Boeing  reported  that  the  time  to  complete  a  wide  range  of  projects 
was  cut  by  91%  (20:93).  These  results  seem  astounding,  yet  are  difficult  to  objectively  substantiate 
in  the  literature.  For  example,  the  IBM  findings  (56%  reduction  in  project  planning  time)  are 
very  subjective.  Nunamaker  et  al  state  that  “actual  man-hour  data  were  aligned  with  anticipated 
time  estimates  [emphasis  added]  based  on  leader  experience  and  historical  precedents”  (30:187). 
He  provides  the  following  amplification: 


10  OptionFinder  is  a  registered  trademark  of  Option  Technology,  Inc.,  GroupSystems  is  a  registered  trademark 
of  Ventana,  Inc.,  TeamFocus  is  a  registered  trademark  of  IBM,  Inc.,  and  VisionQuesi  is  a  registered  trademark  of 
Collaborative  Technologies,  Inc. 
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The  efficiency  of  the  system  as  used  in  this  study  is  an  indication  of  the  relative 
costs  and  benefits  to  the  organization  compared  with  doing  the  same  function  manu¬ 
ally.  It  was  not  possible  to  run  parallel  sessions  with  control  groups  to  measure  efficiency 
directly.  However,  prior  to  use  of  the  [Decision  Support  Center]  facility  and  without 
knowledge  of  automated  support  capabilities,  each  group  leader  was  required  to  recom¬ 
mend  and  document  a  feasible  project  schedule  for  the  accomplishment  of  his  or  her 
group’s  objectives,  based  on  previous  experience  with  similar  projects.  These  schedules 
defined  the  historical  parameters  for  the  projects  and  a  baseline  for  comparison  with 
the  efficiency  of  the  automated  support... After  completion  of  the  project,  expectations 
before  use  of  the  tools  was  compared  with  what  actually  occurred.  Further,  the  output 
from  the  sessions  was  independently  evaluated  by  a  knowledgeable  third  party  to  gain 
a  measure  of  what  it  would  take  to  arrive  at  the  same  level  of  accomplishment  using 
traditional  manual  processes... man-hours  were  saved  in  every  case  recorded,  with  an 
average  per  session  savings  of  55.51%  (30:189). 

It  could  easily  be  argued  that  these  estimates  are  so  highly  speculative  and  subjective  in  nature 
that  they  probably  can  not  be  accepted  at  face  value  without  credible,  independent  substantiation. 
On  the  other  hand,  regardless  of  the  laboratory  research  findings,  the  previous  testimonies  are 
difficult  to  ignore.  The  appeal  to  what  is  being  said  about  GDSS  technology  in  industry  is  that 
these  companies  appear  to  have  discovered  that  potential  time  and  cost  savings,  as  well  as  improved 
effective  decision  making,  are  realizable  goals. 


2.6  Areas  Requiring  Further  Research 

Except  for  a  handful  of  field  studies,  primarily  those  of  Nunamaker  et  al,  GDSS  laboratory 
studies  to  date  have  involved  inexperienced  undergraduate  and  graduate  Business  and  MIS  students 
resolving  fictitious,  unvalidated  tasks  (42:18),  (12:279).  Examples  of  the  tasks  performed  during 
the  experiments  include  (12:279): 

•  A  foreign  embassy  takeover  by  a  terrorist  group  (planning). 

•  Severe  financial  problems  at  a  university  (idea  generation). 

•  A  group  of  people  must  survive  Arctic  conditions  (intellective). 

•  A  student  misconduct  case  at  a  university  (policy  formulation). 

•  A  firm  is  losing  profits  at  the  same  time  sales  are  rising  (problem  finding). 

The  design  of  such  research  is  severely  constrained  by  the  lack  of  experienced  subjects,  not 
the  lack  of  a  desire  to  study  under  more  realistic  conditions.  Obviously,  there  exists  a  genuine  need 
to  conduct  GDSS  research  in  real  business  settings  with  real  managers  solving  real  problems.  There 
is  also  a  need  to  use  similar  GDSS  software  packages  and  similar  environmental  conditions  (setting, 
task,  et  cetera)  for  a  better  comparison  of  results  between  studies.  The  following  comments  made 
by  prominent  GDSS  researchers  and  authors  affirm  these  positions: 
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•  “Additional  research  is  warranted  to  expand  field  observations  and  integrate  aspects  of  field 
and  experimental  research  in  order  to  achieve  a  more  comprehensive  understanding  of  the 
implications  for  organizations...”  (Nunamaker,  Vogel,  et  al,1989),  (30:195). 

•  “There  is  a  real  need  for  field  studies  in  real  organizational  settings  [with  real  managers]” 
(Pinsonneault  and  Kraemer,  1989),  (32:158). 

•  “What  we  still  lack,  however,  is  field  experiments  that  provide  semi-controlled  circumstances 
so  we  can  tell  just  what  is  going  on  in  these  sites”  (Zigurs,  1993),  (44:122). 

•  “We  are  much  more  likely  to  get  a  true  picture  of  overall  GDSS  impact  over  time  through 
field  studies  than  lab  studies”  (Lewis  and  Keleman,  1990),  (21:204). 

•  “Additional  research  is  warranted  to  expand  field  observations  and  integrate  aspects  of  field 
and  experimental  research  in  order  to... achieve  a  more  comprehensive  understanding  of  the 
implications  for  organizations  of  the  adoption  of  Group  Support  Systems”  (Nunamaker  et  al, 
1989),  (30:195). 

•  “Students  were  used  as  subjects  because  it  was  difficult  to  recruit  executive  groups  to  partic¬ 
ipate  in  a  field  experiment”  (Jessup  et  al,  1991),  (18:269). 

•  “The  use  of  students  may  not  have  been  a  wise  choice.  GDSS  technology  is  designed  for  use  by 
persons  who  often  make  complex  decisions  in  groups. ..and  would  have  real-world  experiences 
from  which  to  make  comparisons... The  results  suggest  that  GDSS  is  not  very  effective  when 
introduced  without  a  specific  goal  or  target  audience,  nor  does  it  appear  necessarily  useful  or 
effective  when  used  with  groups  that  do  not  have  a  vested  interest  in  the  task... While  using 
a  more  focused,  real-world  case... would  limit  the  potential  sample  of  subjects,  these  subjects 
would  more  accurately  represent  the  population  that  would  most  likely  use  such  a  system  in 
an  organization”  (Beauclair,  1989),  (1:329,  331). 


2. 7  Summary  of  Existing  Research 

In  the  book  Group  Support  Systems:  New  Perspectives ,n  Alan  Dennis  and  Brent  Gallupe 
recap  the  study  of  GDSS  from  its  inception  to  the  present.  They  conclude  that  the  field  studies 
have  reached  generally  positive  results,  contrary  to  those  of  the  earlier  laboratory  studies  which 
were  largely  inconsistent.  Specifically,  they  conclude  that  GSS  groups  during  the  field  studies 
were  largely  satisfied  with  the  process  and  outcomes,  and  that  users  perceived  that  the  technology 
improved  effectiveness  and/or  efficiency — contrary  to  the  earlier  laboratory  findings  (6:73).  Dennis 
and  Gallupe  highlight  a  popular  premise  which  partially  explains  the  dichotomy  of  the  findings: 
field  studies  involved  larger  groups,  using  managers  or  professionals  performing  complex  tasks 
over  several  days,  and  included  an  active  facilitator;  whereas,  laboratory  studies  usually  involved 
students  performing  relatively  easy  tasks  over  an  hour  or  so  without  the  aid  of  a  facilitator  (6:68). 
Dennis  and  Gallupe  offer  the  following  additional  conclusions  (6:73): 

•  GSS  use  in  the  field  appears  to  improve  performance,  efficiency,  and  user  satisfaction. 
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•  Effects  of  GSS  depend  not  only  upon  aspects  of  the  technology,  but  also  upon  factors  per¬ 
taining  to  the  specific  group  and  task. 

•  The  most  important  aspects  of  a  GSS  situation  are  group  size,  task  complexity,  and  task 
type. 

•  Larger  groups  benefit  more  from  GSS  use  than  do  small  groups. 

•  GSS  “better  fits”  complex  tasks  than  simpler  ones. 

Finally,  Dennis  and  Gallupe  state  that  GSS  use  can  “dramatically  improve  group  performance 
and  satisfaction  for  generation  tasks  [emphasis  added]  ”  but  they  are  “  less  convinced  that  GSS 
technology  can  help  groups  facing  a  choice  task  [emphasis  added]  where  the  objective  is  to  choose 
an  alternative  from  a  prespecified  set.  GSS  may  help,  but  the  evidence  is  not  as  clear”  (6:73). 


2.8  Conclusion 

The  use  of  GDSS  in  any  organization  holds  potentially  great  promise  for  improved  group 
decision  making.  GDSS  reportedly  1)  enhances  the  participation  of  group  members,  2)  allows 
simultaneous  communication  of  ideas,  and  3)  solicits  the  inputs  of  otherwise  timid  or  quiet  par¬ 
ticipants.  The  end  result  is  the  perception  that  GDSS  improves  decision  making,  and  users  are 
satisfied  with  the  process  and  the  results. 

According  to  DeSanctis  and  Gallupe,  the  most  difficult  issue  in  GDSS  research  is  finding 
the  desired  balance  between  decision  quality  and  user  satisfaction.  Often,  quality  is  sacrificed  for 
user  acceptance  or  vice-versa.  Their  position  is  that  these  two  goals  may  be  conflicting  and,  thus, 
“the  long-term  objective  of  GDSS  technology  should  be  to  improve  the  efficiency  and  effectiveness 
of  group  decision  making. ..member  satisfaction  with  the  process  is  necessary  but  not  [a]  sufficient 
condition  for  a  successful  GDSS”  (8:606). 

It  may  be  too  early  to  field  these  systems  on  a  global  scale  until  we  understand  the  technology 
and  the  environments  which  will  benefit  from  its  implementation.  Gray  states  the  following  (16:233- 
234): 


When  these  systems  are  installed  in  industry  and  government,  they  behave  like 
shooting  stars.  They  are  put  in  by  one  senior  executive  and  used  during  his  or  her 
tenure.  However,  as  soon  as  that  individual  is  replaced,  the  system  is  dismantled  or 
falls  into  disuse.  The  major  problem  is  that  at  this  point  we  do  not  know  how  to  use 
these  systems  effectively  [or]  how  to  train  people  (particularly  middle-aged  executives) 
how  to  use  them. ..One  possible  outcome. ..is  that  GDSS  may  be  a  solution  for  which 
there  is  no  known  problem. 

This  relatively  new  and  emerging  technology  is  certainly  not  well-understood.  There  remain 
many  opportunities  for  additional  research.  If  large  corporations,  such  as  IBM  and  Boeing,  are 
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realizing  improved  productivity  and  decision  making,  the  Air  Force  could  potentially  benefit,  too. 
We  should,  however,  apply  GDSSs  where  it  makes  the  most  sense,  when  the  technology  is  mature, 
and  when  we  understand  how  to  use  it.  GDSS  proponents  have  stressed  the  need  to  conduct  field 
research  within  specific  management  environments.  The  Air  Force  should  proceed  in  exploring 
GDSS  technologies  through  research  and  development  efforts  now,  because  this  is  a  rare  opportunity 
for  the  military  to  not  only  capitalize  on  the  advances  being  made  in  this  unique  information  systems 
field,  but  also  to  influence  future  development  of  the  technology. 
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III.  Methodology 

S.l  Introduction 

Researchers  have  studied  GDSS  technology  using  a  variety  of  approaches.  In  total,  these 
studies  have  produced  a  variety  of  mixed  or  contradictory  results.  The  importance  of  documenting 
the  research  approach  and  the  experimental  conditions  is  paramount,  since  much  of  the  literature 
claims  that  results  are  very  much  situation-dependent.  The  purpose  of  this  chapter  is  describe  the 
real-world  environment  in  which  the  current  study  was  conducted  and  the  methodology  that  was 
followed.  Specifically,  this  chapter  describes  the  real-world  situation  where  GDSS  technology  was 
studied,  explains  the  field  experiment  design  that  was  developed  for  this  environment,  and  discusses 
the  statistical  methods  that  were  used  to  analyze  the  collected  data. 

The  importance  of  the  current  study  is  best  understood  in  light  of  the  research  discrepancies 
noted  in  Chapter  II.  This  study  places  significant  experimental  control  on  a  real  organizational 
problem  involving  real  Air  Force  managers  in  a  real  organizational  setting  in  order  to  provide  near- 
experimental  conditions  for  more  objective  measurements,  but  with  enough  flexibility  to  allow  the 
GDSS  to  be  exploited  for  operational  use.  The  setup  of  this  experiment  was  intended  to  provide 
the  necessary  “semicontroP  that  Zigurs  mentioned  was  needed  for  further  research.  Additionally, 
the  research  uses  the  GDSS  software  GroupSystems  which  has  been  frequently  used  in  documented 
field  studies.  This  approach  is  particularly  advantageous,  since  the  experimental  design  directly 
compares  automated  (GDSS)  and  manual  processes  without  the  need  to  estimate  comparisons 
based  on  historical  information  or  previous  personal  experiences. 

“An  experiment  is  the  premier  scientific  methodology  for  establishing  causation”  and  is  es¬ 
tablished  by  performing  the  following  activities  (9:419): 

1.  Selecting  relevant  variables  for  study. 

2.  Specifying  levels  of  the  various  treatments. 

3.  Controlling  the  experimental  environment. 

4.  Choosing  the  overall  experimental  design. 

5.  Selecting  and  assigning  subjects  to  participate  in  the  experiment. 

6.  Pilot  testing,  revising,  and  testing  the  experiment. 

7.  Analyzing  the  resulting  data. 

Although  not  necessarily  discussed  in  precisely  this  order,  the  current  experiment  incorporated 
these  activities,  to  the  fullest  extent  possible  for  a  field  study. 
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3.2  Current  Study  Research  Objectives 


The  purpose  of  this  study  was  to  conduct  GDSS  research  in  an  Air  Force  environment  to 
objectively  measure  and  evaluate  both  GDSS  effects  and  structure  (facilitation)  effects  in  real- 
world  group  decision  making  processes.  The  study  evaluated  two  objectively  measured  performance 
outcomes:  1)  effectiveness  in  terms  of  consensus,  and  2)  efficiency  in  terms  of  decision  speed. 
The  study  also  evaluated  eight  subjectively  measured  secondary  outcomes  relating  to  the  user’s 
satisfaction  and  perceptions.  As  stated  in  Chapter  I  and  repeated  here  for  completeness,  the 
research  objectives  were  identified  by  the  following  sub-objectives: 

1.  GDSS  Effects:  Performance  Outcomes 

(a)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
decision  speed. 

(b)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
consensus. 

2.  GDSS  Effects:  Secondary  Outcomes 

(a)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  satisfaction  with  the  decision  making  process. 

(b)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  satisfaction  with  the  results  of  the  decision  making  process. 

(c)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  perception  of  task  difficulty. 

(d)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  perception  of  the  group’s  ability  to  stay  task-focused. 

(e)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  perception  of  productivity  in  terms  of  time  efficiency. 

(f)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  perception  of  productivity  in  terms  of  ability  to  reach  consensus. 

(g)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  perception  of  the  group’s  level  of  conflict. 

(h)  Compare  facilitated  GDSS  and  non-GDSS  (facilitated  manual)  groups  with  respect  to 
the  user’s  perception  of  the  level  of  process  structure. 

3.  Structure  Effects:  Performance  Outcomes 

(a)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  decision  speed. 

(b)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  consensus. 

4.  Structure  Effects:  Secondary  Outcomes 

(a)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  satisfaction  with  the  decision  making  process. 
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(b)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  satisfaction  with  the  results  of  the  decision  making  process. 

(c)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  perception  of  task  difficulty. 

(d)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  perception  of  the  group’s  ability  to  stay  task-focused. 

(e)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  perception  of  productivity  in  terms  of  time  efficiency. 

(f)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  perception  of  productivity  in  terms  of  ability  to  reach  consensus. 

(g)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  perception  of  the  group’s  level  of  conflict. 

(h)  Compare  facilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect 
to  the  user’s  perception  of  the  level  of  process  structure. 

These  sub-objectives  were  chosen  because  they  were  fairly  common  to  most  of  the  previous 
research  and  could  be  readily  adapted  to  the  real-world  situation.  Including  so  many  sub-objectives 
allows  others  to  easily  compare  the  current  results  with  previous  laboratory  and  field  studies.  More 
importantly,  the  research  provides  information  for  comparison  with  similar  future  field  studies 
which  provide  a  reasonable  amount  of  control  and  objectivity. 

As  part  of  the  experimental  design,  the  subjective  portion  of  the  research  sought  to  measure 
whether  groups  perceived  improved  productivity  in  terms  of  time  efficiency  and  consensus  (i.e., 
performance  outcomes),  without  regard  to  the  objectively  measured  outcomes.  This  approach  is 
important,  since  GDSS  users  may  perceive  an  improvement  in  performance  (or  lack  of  improvement) 
when,  in  reality,  performance  may  not  have  improved.  This  perception  may  lead  them  to  potentially 
like  or  dislike  the  tool  even  though  reality  contradicts  their  feelings. 


3.3  Field  Experiment  Environment  Description 

The  Air  Force  conducted  a  two-day  conference  in  July  1992  to  develop  an  environmental 
education  strategy.  This  conference  provided  an  opportunity  to  observe  and  evaluate  small  groups 
making  choice-type  decisions.  Additionally,  this  event  provided  an  environment  where  experimental 
controls  could  be  coupled  with  a  real  world  decision  making  process.  To  explain,  the  Deputy 
Assistant  Secretary  of  the  Air  Force  for  Environment,  Safety,  and  Occupational  Health  tasked 
the  Air  Force  Institute  of  Technology  (AFIT)  and  the  USAF  School  of  Aerospace  Medicine  (i.e., 
organizations  responsible  for  providing  environmental  education  to  USAF  personnel)  to  develop 
the  USAF  Environmental  Education  Master  Plan.1  This  task  first  required  the  identification  of 

’This  plan  is  a  comprehensive,  integrated  ervironmental  education  and  training  plan  for  all  USAF  professions. 
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critical  education  categories  and  later  the  levels  of  education/training  needed  for  each  particular 
Air  Force  profession  or  career  field. 

At  the  first  working  group  in  April  1992,  key  environmental  professionals  met  at  Brooks  AFB 
in  San  Antonio,  Texas  and  identified  categories  of  USAF  professions  (reference  Table  3.1)  and 
unique  environmental  tasks  (reference  Table  3.2)  which  must  be  considered  for  training  in  these 
USAF  occupations.2  The  next  step  in  this  process  required  the  sponsors  to  identify  the  appropriate 
level  of  training  for  each  environmental  task  against  each  specific  USAF  profession  using  a  small 
group  decision  making  process.  This  small  group  decision  making  process  was  evaluated  by  the 
current  research. 

Armstrong  Laboratories  and  the  sponsoring  organizations  collectively  agreed  to  conduct  a 
variant  of  the  Nominal  Group  Technique/Delphi  process  to  select  the  levels  of  training.  Approxi¬ 
mately  40  participants  met  in  a  large  group  setting  to  discuss  the  meaning  and  specific  examples 
of  the  51  environmental  tasks  identified  in  the  April  92  Environmental  Working  Group  (reference 
Table  3.2).  Thirty-five  members  within  this  group  later  divided  into  seven  groups  of  five  members 
each  to  discuss  and  evaluate  the  level  of  training  needed  for  each  of  the  51  tasks  for  each  career 
field  represented  in  the  group.  Ideally,  each  small  group  included  a  representative  for  each  of  four 
unique  career  fields  to  be  evaluated. 

These  small  groups  were  randomly  selected  to  receive  different  treatments  in  the  manner 
in  which  they  conducted  their  meetings.  The  primary  treatment  was  use  of  GDSS  technology; 
however,  several  other  variables  were  introduced  in  facilitated  manual  group  settings  in  order  to 
evaluate  the  effect  of  structure.  Following  evaluation  of  the  204  distinct  decisions  (51  tasks  times 
4  career  fields),  the  small  groups  returned  to  the  large  group  setting  to  discuss  and  resolve  specific 
tasks  for  which  the  group  did  not  reach  consensus.  The  overall  goal  in  this  process  was  to  reach 
consensus  on  the  maximum  number  of  tasks  possible  during  the  small  group  sessions  and  to  reach 
full  consensus  on  the  remaining  tasks  in  the  large  group.  The  research  focused  only  on  the  small 
group  process  in  accordance  with  the  objectives  stated  previously.  A  task  evaluation  scale  shown 
in  Figure  3.1  identified  the  levels  of  training  which  could  be  assigned  for  each  environmental  task 
in  each  career  field.  Recalling  McGrath’s  Task  Circumplex,  this  type  of  task  was  a  choice  decision 
making  task  where  participants  made  decisions  for  issues  which  have  no  right  answer. 

This  conference  lasted  two  days.  Initially,  on  the  morning  of  the  first  day,  all  participants  met 
in  a  large  group  setting  to  discuss  and  further  define  each  of  51  environmental  tasks.  Subsequent 
to  that  meeting,  on  the  afternoon  of  the  first  day  and  the  morning  of  the  second  day,  the  small 

JThe  terms  “occupations,"  “professions,”  and  “career  fields"  are  used  interchangeably  in  this  document. 
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Table  3.1.  USAF  Professions  Categorized  by  Priority  (34:Atch  2). 


|  CATEGORY  I  | 

Bioenvironmental  Engineering 

Civil  Engineering/Environment  Engineering 

Environmental  Staff  Judge  Advocate 

Public  Affairs 

Logistics 

Installation  &  Group  Commanders 

System  Program  Office  Hazardous  Materials  Personnel 

Fire  Fighters 

Base  Disposal  Personnel 

Flight  Medicine 

|  CATEGORY  II  | 

Corrosion  Control 

Military  Public  Health 

Toxicologists 

Systems  Safety  > 

Disaster  Preparedness 

Medical  Treatment  Facility  Commanders 

Materials  Engineering 

Process  Engineering 

Manufacturing  Engineering 

Contract  Inspectors 

Procurement 

Fuels  Personnel 

Maintenance  Quality  Assurance/Control 

Industrial  Supervisors 

Plating  Shop  Personnel 

Vehicle  Maintenance 

Waste  Treatment 

Defense  Reutilization  A  Management  Office 

Morale,  Welfare,  &  Recreation 

Medical  Treatment  Facility  Plant  Managers 

Medical  Logistics 

Photo  Laboratory  Personnel 

Pest  Control 

Contracting 

j  Industrial  Shop  Worker  j 

_ CATEGORY  III _ 

Ground  Safety 
Security  Police 

Army  Air  Force  Exchange  Service  Warehouse  Worker 
Real  Estate 

Defense  Commissary  Agency 
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Table  3.2.  USAF  Environmental  Tasks  (34:Atchs  3  ic  4). 


ENVIRONMENTAL  TASK 


Incident  Response 


Huatdoai  Material*  (HAZMAT)  Management 


Buiroimtaiti  Mo*»lori*| 


Hamardoa*  Abatement 


Recognition  of  Environmental  Problem* 


Hazardous  Material*  Traiaiag 


Watte  Management  Recycling  Treat  meat  Sc  Disposal 


Environmental  Sampling 


Environmental  Audits/Tracking 


Hazardous  Communication*  (HAZCOM) 


Inatallation  Restoration  Program  (IRP) 


Permit  Management 


Health  Assessment 


Regulatory  Requirements  Identification 


Material  Substitution 


Risk  Analysis  Management 


Risk  Communications 


Plan*  Review. Environmental  Aspects 


Environmental  Project  Design 


Environmental  Contract* 


Community  Involvement 


Notice  of  Violations  (NOV)  Tracking 


Review  Pending  Laws 


Management  Installation  Geographically 
Separated  Units  (GSUs) 


Environmental,  Safety,  and  Occupational 
Health  (ESOH)  Orientation 


Revise  Environmental  Technical  Data 


Clinical  Evaluation  and  Investigation 


Crossfeed  Lessons  Learned 


Real  Estate  Management 


Design  Trade-Off  Analysis 


Program,  Planning,  and  Budgeting _ 


Process  Assessment 


EnvironmentaJ  Policy  Management _ 


Project  Management 


Natural  and  Cultural  Resource  Conservation 


Dissemination  of  Environmental  Policy  to 
Civil  Engineering  (CE)/Logi*tlc*( LO)/Medical(SQ) _ 


Policy  and  Plan*  Management 


Contract  Management 


Functional  Responsibilities 


Integrate  into  Position  Descriptions  (PDs),  Safety  Training 
Standards  (STS),  Safe  Sc  Proper  Procedures  (SPP) 


Advocacy  Sc  Resource  Allocation 


Information  Management 


Records  Management 


Reuse,  Recover,  Sc  Recycling  _ 


Technology  Assessment 


Strategic  Planning  _ 


Interagency  Sc  Industrial  Cooperative  Efforts 


Understand  Acquisition  Sc  Logistic* 


Life  Cycle  Cost  Analysis 


Cost  Benefit  Analysis 


Computer  Operations  Traiaiag 


!  How  Important  is  this  Task  to  the  Career  Field? 

Level  5:  Task  is  an  essential  responsibility  of  this  career  field. 

Level  4:  Career  field  must  understand  and  be  able  to  perform  this  task. 

Level  3:  Career  field  must  understand  this  task. 

Level  2:  Career  field  should  have  some  knowledge  of  this  task. 

Level  1 :  Career  field  never  performs  this  task  and/or  has  no  need  for  additional 
Saining  and  education  in  this  task. 

Figure  3.1.  Task  Evaluation  Scale:  Environmental  Training  Levels. 

groups  completed  the  decision  making  process  of  choosing  what  they  felt  were  appropriate  training 
levels  for  each  specific  AF  career  field.  Finally,  on  the  afternoon  of  the  second  day,  the  large  group 
reconvened  to  review  the  outcomes  of  the  small  group  processes. 

Three  of  these  small  groups  were  exposed  to  GDSS  technology,  while  the  other  four  groups 
were  exposed  to  only  manual  (facilitated  or  unfacilitated)  processes.  The  current  experiment  cap¬ 
tured  measurements  from  the  small  group  processes. 

The  objective  of  the  research  was  to  evaluate  GDSS  and  process  structure  effects  to  the  group 
decision  making  process.  In  order  to  provide  a  more  robust  study,  a  multi-methodological  research 
effort  was  undertaken:  objective  measurements  of  the  performance  outcomes  of  decision  making 
speed  and  consensus  were  taken  by  group  facilitators  or  designated  team  leaders,3  while  subjective 
measurements  of  the  secondary  outcomes  (i.e.,  individual  satisfaction  levels  and  perceptions)  were 
obtained  through  the  use  of  post-process  questionnaires.  A  pre-process  questionnaire  was  also  used 
to  measure  the  participant  demographic  characteristics. 

The  small  groups  consisted  of  five  members:  one  individual  to  represent  each  of  four  career 
fields  and  one  environmental  professional.  All  five  members  participated  in  the  small  group  discus¬ 
sions  and  voted  for  the  appropriate  training  levels.  All  five  people  attempted  to  reach  full  consensus 
on  the  environmental  task  training  levels.  Approximately  two  hours  were  allocated  for  each  session. 
The  small  groups  were  instructed  to  complete  two  career  fields  each  day.  Any  tasks  for  which  the 
groups  did  not  achieve  consensus  were  brought  back  to  the  large  group  the  afternoon  of  the  second 
day  for  further  discussion  and  resolution.  Small  groups  using  GDSS  technology  used  a  decision 
room  in  a  face-to-face  environment  as  illustrated  by  the  taxonomy  provided  in  Figure  3.2. 

3 These  were  the  Environmental  Professionals  for  the  unfacilitated  manual  groups. 
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Figure  3.2.  USAF  Experiment  Taxonomy  (16:239). 


3-4  Group  Size  and  Individual  Assignments 

This  experiment  involved  a  small  group  process.  Small  groups  have  been  typically  classified 
in  the  literature  as  those  with  fewer  than  five  members;  however,  there  are  differing  opinions 
as  to  exactly  how  many  members  constitute  a  small  or  large  group.  For  example,  George  et  al 
state  that  large  groups,  greater  than  four  members,  tend  to  get  more  efficiency  and  effectiveness 
gains  from  GDSS  than  small  groups  (i.e.,  four  or  fewer  members),  while  others  clearly  state  that 
large  groups  are  primarily  those  with  at  least  five  members  (14:24).  Essential  research  is  needed 
to  target  the  group  size  most  pertinent  to  that  size  of  groups  which  meet  in  real  organizations 
Watson  et  al  quote  a  1986  article  in  Datamation4  which  states  that  “the  average  number  of  people 
attending  an  organizational  meeting  is  only  five”  (43:466).  This  group  size  condition  is  consistent 
with  the  experimental  design  of  the  current  research;  however,  it  should  be  noted  that  group  size 
was  determined  primarily  by  the  sponsoring  organizations.  The  researcher  sought  to  maintain  the 
groups  participating  in  the  study  at  a  uniform  size. 

Subjects  participating  in  an  experiment  should  be  representative  of  the  population  being 
studied  in  order  to  assess  generalization  of  the  results  obtained  (9:421-422).  Since  the  subjects 

‘Refers  to  “Hardware:  Offline,”  Datamation,  32:  109  (May  15,  1986). 
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were  drawn  from  many  USAF  professions,  they  represented,  at  least  functionally,  a  cross  section  of 
the  mainstream  USAF. 

The  researcher  was  concerned  that  the  different  individual  backgrounds  of  the  participants 
could  potentially  confound  the  experiment’s  results.  This  concern  is  not  unique  to  this  study.  In 
any  experiment,  the  researcher  attempts  to  negate  or  reduce  the  effects  due  to  confounding  vari¬ 
ables.  This  practice  can  be  accomplished  primarily  by  two  means:  either  controlling  the  variable 
or  matching  the  variable  with  equal  distributions  across  all  groups  (9:157).  Whenever  these  ap¬ 
proaches  are  not  feasible — as  in  the  current  study — the  researcher  uses  the  process  of  randomization. 
Emory  states  that  “randomization  is  the  basic  method  by  which  equivalence  between  experimental 
and  control  groups  is  determined. ..random  assignment  to  experimental  and  control  groups  is  the 
basic  technique  by  which  two  groups  can  be  made  equivalent”  (9:158).  What  this  means  is  that 
through  random  assignment  each  group  would  supposedly  receive  its  fair  share  of  different  factors. 
Deviations  experienced  because  of  personal  differences,  rather  than  the  experimental  treatment, 
would  be  randomly  distributed  among  all  groups.  Their  effects  on  the  dependent  variables  would 
be  random,  thereby  minimizing  “extraneous  noise”  and  improving  “sensitivity  of  measuring  the 
hypothesized  relationship”  (9:158,  421-422).  The  participants  were  assigned  to  small  groups  by 
the  sponsoring  organizations.  Individual  career  specialties  served  as  the  primary  criteria  for  these 
group  assignments.  The  environmental  representatives  were  also  assigned  to  groups  by  the  sponsor¬ 
ing  organizations  where  their  backgrounds  and  expertise  best  matched  the  career  fields  represented 
in  a  particular  group. 

Finally,  the  sponsors  did  not  select  specific  people  to  attend  the  conference.  This  decision 
was  left  to  the  individual  commanders  of  the  organizations  represented  at  the  conference.  Another 
precaution  was  undertaken.  In  an  experiment,  whenever  a  variable  can  neither  be  controlled  nor 
eliminated,  at  a  minimum  it  should  be  measured.  Since  personnel  assignments  to  the  small  groups 
were  not  completely  random,  a  pre-process  questionnaire  was  administered  to  all  group  members. 


3.5  Group  Participant  Demographics 

Prior  to  the  small  group  process  on  the  first  day,  all  participants  voluntarily  completed  a 
Pre-Process  Questionnaire  identical  to  that  shown  in  Figure  3.3.  The  questionnaire  served  two 
purposes.  First,  it  was  used  to  characterize  the  demographics  of  the  groups  in  terms  of  age,  gender, 
rank,  job  experience,  and  education  to  show  essentially  equivalence  across  groups.  Second,  it  was 
used  to  determine  the  “computer  literacy”  of  the  participants  and  to  note  if  any  particular  group 
was  computer-averse. 


3-9 


Rank _  Age _  Gender  Male  /  Female 

How  long  have  you  been  assigned  to  your  present  career  field? _ Years 

How  long  have  you  been  assigned  to  your  present  organization? _ Years 


Do  you  use  a  computer  regularty  at  work?  Yes  /  No 

Do  you  own  a  personal  computer  for  use  at  home?  Yes  /  No 

Have  you  taken  any  computer-related  training  classes  (work  or  home)?  Yes  /  No 

Please  indicate  the  highest  level  of  education  you  have  attained  (mark  only  one): 

□  High  School 

□  Some  College  Courses  Taken 

□  Undergraduate  Degree 

Q  Some  Graduate  Level  Courses 

□  Graduate  Degree 

□  PhD  or  MD 

Please  answer  the  following  question  using  the  scale  provided  below  (circle  only  ONE  number): 
How  comfortable  are  you  with  using  a  computer? 

EivsnMy  Madareiay  0arety  Barely  MMaMly  Eiwsay 

Uncam  far  were  Uwntrfia  UneamftvtMM  B— wtm  CamiorwtM  Camilla  CamtoWa 

I— I — I — I — I — I— I 

How  many  members  of  your  group  have  you  worked  with  previously? _ 


Figure  3.3.  Pre-Process  Questionnaire. 


S.6  GDSS  Equipment  and  Facilities  (26) 


The  GDSS  experiments  were  conducted  at  Building  434  on  Wright-Pattereon  AFB,  Ohio. 
This  facility  is  managed  by  the  USAF  Armstrong  Laboratories,  Human  Resources  Directorate, 
under  the  direct  control  of  the  Logistics  Research  Division  (AL/HRG).  The  GDSS  facility  was 
configured  for  two  distinct  group  decision  making  sessions.  These  two  areas  were  virtually  identical 
decision  rooms  of  approximately  equal  size,  each  with  five  networked  GDSS  workstations.  This 
configuration  allowed  simultaneous  operation  of  two  decision  rooms  to  support  the  experimental 
design  used  in  this  study. 

Each  facility  was  configured  as  shown  in  Figure  3.4  with  five  participant  workstations,  a 
technographer’s  workstation,  a  facilitator’s  podium,  and  an  overhead  projector/liquid  crystal  dis¬ 
play  with  a  portable  public  viewing  screen.  The  facility  dimensions  were  approximately  20  by  15 
feet.  The  rooms  were  very  typical  in  decor  of  other  USAF  conference  facilities  with  the  addition  of 
sound-proof  wall  coverings,  task  and  wash  lighting,  and  a  two-way  mirror  for  visual,  non-disruptive 
observations. 

Each  of  the  five  participant’s  and  the  technographer’s  workstations  were  connected  via  a 
baseband  local  area  network.  The  workstations  ran  in  a  client/server  mode  with  the  network  server 
running  Novell’s  Netware  version  3.11  operating  system. 

The  fileserver  was  a  Dell  316LT  (80386SX  at  16  MHz)  laptop  computer  with  8  Mbytes  of  RAM 
and  120-Mbyte  hard  disk  drive.  Each  participant  workstation  was  either  a  Dell  316LT  laptop  or  an 
ABSEC  NP-913  notebook  (80386SX  at  20  MHz)  computer.  Each  workstation  had  the  capability 
to  run  independently  using  either  MS-DOS  versions  4.0  or  5.0.  All  the  participant  workstations 
had  4  Mbytes  of  RAM  and  used  an  externally  connected  14-inch  VGA  color  monitor. 

The  technographers’  workstations  were  either  a  Dell  316LT  laptop  or  an  upgraded  Zenith 
386DX  (25  MHz)  desktop  personal  computer.  The  projection  capability  consisted  of  a  3M  over¬ 
head  projector  with  a  VGA-capable  IN  FOCUS  liquid  crystal  display,  connected  directly  to  the 
technographer’s  workstation,  and  a  portable  overhead  projection  screen.  The  laboratories’  GDSS 
network  is  designed  to  be  relocatable  in  a  short  amount  of  time.5 

5AL/HRG's  system  design  provides  the  flexibility  to  support  GDSS  operations  at  any  USAF  user’s  facility.  This 
arrangement  may  be  one  of  only  a  handful  of  systems  within  the  GDSS/GSS  research  community  which  possesses 
this  portability. 
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Portable  Projector  Screen 


Figure  3.4.  GDSS  Facility  Schematic  for  the  Field  Experimentation. 
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5.7  Non-GDSS  Facilities 


Facilitated-Manual  groups  met  separately  in  three  individual  conference  rooms,  two  at  the 
Hope  Hotel  Conference  Facility  on  Wright-Patterson  AFB  and  one  at  Armstrong  Laboratories. 
Groups  1  and  2  used  the  conference  room  at  the  laboratory  when  not  using  a  GDSS  decision 
room.  In  this  manner,  no  group  traveling  to  the  laboratory  had  prior  knowledge  of  the  specific 
experimental  condition  they  would  receive.  The  conference  rooms  at  both  the  laboratory  and  the 
Hope  Hotel  were  similar  in  accommodations:  one  conference  table  and  enough  upholstered  chairs  for 
the  participants  and  the  facilitator.  Groups  following  the  traditional  unfacilitated  manual  process 
met  in  separate  areas  of  the  large  conference  room  or  relocated  to  other  areas  within  the  conference 
center  as  they  desired.  There  were  no  restrictions  regarding  where  and  how  these  unfacilitated 
manual  process  groups  met,  except  for  a  few  administrative  details  which  are  discussed  later  in  this 
chapter. 


3.8  GDSS  Software  Description  and  Procedures  (40:GM126) 

There  has  been  significant  criticism  in  the  literature  regarding  the  comparison  of  GDSS  studies 
which  used  different  software  packages.  The  researcher  for  the  current  study  chose  to  use  Ventana 
Corporation’s  GroupSystems  V  for  the  GDSS  portion  of  Experiment  1  for  three  reasons.  First, 
GroupSystems6  has  been  used  almost  exclusively  as  the  tool  of  choice  in  documented  GDSS  field 
studies  to  date.  This  fact  enhances  the  ability  to  make  comparisons  of  findings  with  published 
studies.  Second,  GroupSystems  V  offers  an  excellent  software  utility  called  Group  Matrix  which 
is  useful  for  compiling  and  publicly  showing  group  voting  results  and  was  well-suited  for  this 
experiment’s  application.  Group  Matrix ,7  as  the  name  implies,  is  a  matrix  of  rows  and  columns 
which  allows  groups  to  assign  relationships  between  row  and  column  entities.  Figure  3.5  captures 
a  generic  layout  of  the  screen  as  it  would  appear  to  an  individual  user  and  the  group  at  large. 
Third,  Armstrong  Laboratory  owned  a  research  license  to  this  software  package  and  was  interested 
in  conducting  research  using  this  specific  software  tool. 

For  this  particular  experiment,  the  rows  in  Group  Matrix  represented  the  four  unique  career 
fields  assigned  to  each  specific  group  for  evaluation.  The  columns  represented  each  of  the  51  envi¬ 
ronmental  tasks  that  were  identical  for  all  GDSS  groups.  The  cell  for  each  row/column  intersection 
contained  a  selected  training  level.  Each  individual’s  screen  showed  his/her  latest  vote  (i.e.,  the 
training  level)  for  a  specific  career  field/environmental  task.  The  public  screen  showed  the  average 

6  GroupSytltmt  is  a  registered  trademark  of  Ventana  Corporation. 

'  Group  Matrix  is  a  registered  trademark  of  Ventana  Corporation. 
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Onup  Matrix 


Taakl 

Teak  2 

Taek  3 

Teak  4 

7 

Career  Raid  t 

Citht  Field  2 

Career  Field  3 

Career  Flew  4 

L 

1 

Task  51 

Figure  3.5.  Example  of  a  Group  Matrix  Screen  (40:GMll). 


of  the  votes  (rounded  to  the  nearest  integer)  of  all  group  members  for  each  career  field/task  inter¬ 
section.  A  cell  selector,  indicated  in  Figure  3.5  as  a  double-lined  border  box,  showed  the  current 
cell  selected — this  is  how  individuals  selected  a  career  field/task  intersection  to  cast  votes. 

The  public  screen  was  generated  from  the  technographer’s  workstation.  In  coordination  with 
the  facilitator,  the  technographer  controlled  when  votes  were  cast  and  when  the  public  screen  was 
refreshed  to  show  the  latest  vote  tally.  Once  a  group  reached  consensus  for  a  given  task,  that  cell 
was  highlighted  automatically  in  green  on  the  technographer’s  display  and  the  public  screen.  Prior 
to  reaching  consensus  (or  if  non-consensus),  cells  which  had  been  voted  against  were  highlighted  in 
red. 

GroupSystems  allows  adjustable  thresholds  for  consensus  determination.  Normally  a  threshold 
setting  of  one  (the  default)  would  show  green  if  all  responses  were  within  one  standard  deviation 
of  the  group’s  mean,  assuming  a  normal  distribution.  For  the  current  experiment,  the  consensus 
threshold  was  set  at  zero  (i.e.,  all  five  votes  had  to  be  identical),  since  the  small  group  objective 
was  to  reach  absolute  consensus  on  each  task  for  all  career  fields.  Since  Group  Matrix  updates  the 
public  screen  automatically — posing  a  potential  bias  to  participants  who  lag  behind  in  the  voting 
process — the  technographer  used  the  Cell  Summary  pop-up  screen  in  order  to  hide  the  intermediate 
voting  results.  Once  the  facilitator  had  confirmed  that  all  individuals  had  voted,  the  technographer 
removed  the  pop-up  screen  to  reveal  the  group’s  vote.  If  consensus  were  reached,  the  facilitator 
recorded  the  outcome  and  moved  the  group  on  to  vote  on  the  next  task.  If  not,  the  technographer 
brought  up  the  Cell  Summary  pop-up  screen  which  showed  the  distribution  of  votes  for  each  task 
by  training  level.  In  this  way,  participants  could  see  the  actual  distribution  of  votes  and  discuss  the 
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rationale  for  the  differences.  This  process  was  repeated  until  consensus  was  reached  or  the  group 
had  voted  three  times  for  a  specific  career  field/task  training  level,  whichever  occurred  first. 

3.8.1  Training  for  Groups  Using  GDSS.  Lewis  et  al  state  that  “groups  should  not  be  ex¬ 
pected  to  learn  a  new  tool  at  the  same  time  they  are  trying  to  solve  a  difficult  problem,  even  if 
the  tool  appears  quite  simple... a  ‘hands-on’  dry  run  is  important  so  the  group  members  can  avoid 
worrying  about  learning  the  tool  and  focus  on  the  problem”  (21:204).  George  et  al  claim  that 
new  GDSS  groups  can  “productively  use  automated  [GDSS]  support  with  five  minutes  or  less  of 
instruction”  (14:23). 

Operation  of  the  GroupSystems  software  for  the  participants  was  relatively  straightforward. 
The  group  members  needed  to  understand  how  to  select  a  career  field/task  intersecting  cell,  how  to 
pop-up  a  screen  which  textually  described  each  training  level,  and  how  to  cast  their  votes.  They  also 
needed  to  understand  how  the  group  public  screen  differed  from  their  own  personal  computer  screen 
and  what  procedures  would  be  followed  in  using  the  software.  These  topics  were  fully  addressed 
at  the  beginning  of  each  GDSS  session  as  members  spent  10-15  minutes  rehearsing  a  mock  voting 
exercise  while  using  a  matrix  identical  to  that  of  the  real  group  process.  The  individuals  were 
instructed  to  ignore  task  and  career  field  meanings  during  the  trial  runs — they  were  merely  to 
practice  the  voting  mechanism  without  regard  for  the  subject  matter. 

3.9  Research  Personnel 

Five  facilitators  were  selected  from  the  staff  of  instructors  at  the  AFIT  School  of  Logistics 
and  Acquisition  Management.  These  facilitators  were  graduate-level  instructors  who  were  fairly 
experienced  in  facilitating  group  discussions.  The  two  facilitators  used  for  the  GDSS  processes  were 
comfortable  using  GroupSystems,  as  they  had  been  exposed  to  the  software  during  pretest  training 
for  the  current  research  and  other  academic/professional  encounters.  They  were  not,  however, 
experienced  in  the  technical  aspects  of  the  software’s  operations.  The  technographers  provided  the 
necessary  expertise  to  operate  the  software. 

The  technographers  were  staff  members  of  the  USAF  Armstrong  Laboratory  at  Wright- 
Patterson  AFB,  Ohio.  These  individuals  were  highly  trained  in  numerous  Groupware  products 
and  were  very  familiar  with  the  specific  technical  aspects  of  operating  GroupSystems.  They  were 
primarily  silent  members  during  the  actual  small  group  processes;  however,  they  assisted  with  the 
GDSS  training  prior  to  each  group  session  and  assisted  the  facilitators  by  operating  the  system  and 
answering  any  technical  questions. 
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The  environmental  representatives  were  not  formal  members  of  the  research  team;  however, 
they  served  as  team  leaders  for  the  unfacilitated  manual  groups.  In  this  capacity,  they  accomplished 
record-keeping  of  interval  times  and  final  consensus  levels  during  the  traditional  unfacilitated  man¬ 
ual  small  group  sessions. 

3.10  Experiment  Procedures 

All  seven  groups  involved  in  this  study,  regardless  of  experimental  condition,  were  required  to 
determine  training  levels  for  51  environmental  tasks  for  each  of  four  career  fields  represented  in  their 
small  groups  using  a  voting  process.  The  primary  objective  of  this  small  group  decision  making 
process  was  to  reach  consensus  on  as  many  of  the  204  tasks  as  possible.  Those  tasks  for  which  a 
group  did  not  achieve  consensus  would  be  deferred  to  the  large  group  for  resolution.  Individuals 
did  not  know  in  advance  to  which  small  group  they  would  be  assigned,  nor  did  they  have  prior 
knowledge  that  group  process  experimentation  would  be  conducted.  The  facilitator  for  the  large 
group  session  provided  some  general  information  at  the  beginning  of  the  conference  similar  to  that 
shown  in  Figure  3.6. 

The  procedures  for  all  facilitators  (GDSS  and  Facilitated-Manual)  were  straightforward,  and 
strictly  enforced.  The  goal  in  providing  stringent  procedures  was  to  ensure  that  the  level  of  struc¬ 
ture  in  both  GDSS  and  FAC-MAN  groups  was  essentially  identical.  The  facilitators  followed  the 
instructions  in  Figure  3.7  which  required  them  to  vote  sequentially  (one  career  field  and  one  task 
at  a  time)  and  vote  no  more  than  three  times,  regardless  of  whether  or  not  the  group  reached  con¬ 
sensus.  This  restriction  was  a  compromise  between  the  research  team  and  the  conference  sponsors 
to  ensure  that  the  process  moved  along  in  a  timely  manner  while  still  accomplishing  the  research 
objectives. 

To  ensure  the  voting  procedures  were  standardized,  the  facilitators  sought  an  initial  vote  for 
each  task  prior  to  any  group  discussion.8  Time  limits  for  discussion  were  not  imposed;  however, 
facilitators  were  encouraged  to  keep  the  process  moving.  A  second  vote  was  taken.  If  consensus 
was  not  met,  a  third  vote  was  taken  without  additional  discussions.  This  process  attempted  to 
avert  needless  flip-flopping  or  swapping  of  two  opposing  votes  (i.e.,  a  “2”  swaps  with  a  “3”  and 
vice-versa).  Facilitators  recorded  both  interval  times  (every  ten  tasks)  and  the  distribution  of  votes. 
The  distribution  of  votes  was  required  to  evaluate  consensus  during  the  post-experiment  analysis. 
Finally,  facilitators  were  requested  to  provide  additional  comments  concerning  group  activities 

8This  procedure  follows  the  Delphi  and  NGT  processes. 
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Script  for  Large  Group  Facilitator 


Today,  as  part  of  the  Small  Group  Discussion  and  Voting  activities,  some  of  our  AFIT 
people  will  be  studying  different  aspects  of  small  group  processes.  Your  part  in  this  effort 
is  minimal,  but  very  important. 

All  of  you  will  be  asked  to  fill  out  three  very  brief  questionnaires  which  will  help  us  to 
characterize  the  background  of  this  whole  group  and  also  obtain  some  of  your 
perceptions  as  they  relate  to  the  small  group  processes  you’ll  be  experiencing. 

Please  participate  fully  in  these  very  brief  survey  instruments.  Be  honest  and  objective. 
Your  feedback  is  essential  towards  improving  workshops  such  as  this  one. 

Finally,  some  of  you  will  be  meeting  in  different  locations  today  and  tomorrow  for  the 
small  group  activities.  You  have  been  provided  with  a  list  showing  the  group  to  which 
you  are  assigned  and  where  and  when  to  meet.  For  those  identified  to  participate  in  an 
off-site  location,  please  note  that  you  will  be  meeting  a  van  behind  the  conference  center 
approximately  15  minutes  earlier  for  each  scheduled  small  group  session. 

Please  arrive  promptly  at  your  group’s  designated  location  each  day.  And  thanks  for 
allowing  us  to  use  this  occasion  to  study  group  processes. 


Figure  3.6.  Research  Related  Script  for  the  Large  Group  Facilitator. 
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instructions  for  Facilitators 


Procedures  for  Group  Process 

Consensus  voting  and  subsequent  discussions  (if  needed)  must  be  performed  individually  for  each  task 
per  career  field  (i.e.,  one-at-a-time).  Do  not  vote  for  multiple  tasks  all  at  once. 

Your  first  vote  for  each  task  MUST  precede  any  discussion  of  that  task,  except  for  clarification  of  the  task 
description,  if  needed. 

Do  not  vote  more  than  THREE  TIMES  for  any  given  task.  VOTE-DISCUSS-VOTE-FINAL  VOTE. 

If  consensus  is  not  reached  on  the  third  vote,  annotate  the  overhead  acetate  with  the  range  of  votBs  to 
show  degree  of  nonconsensus.  DO  NOT  DEVIATE  FROM  THIS  PROCEDURE. 

Make  sure  that  FIVE  people  vote  -  that  includes  the  Environmental  Professional. 

Procedures  for  Data  Collection 

Note  GROUP  NUMBER,  CAREER  FIELD,  and  DATE  on  each  Data  Collection  Form  (IMPORTANT). 

Record  START,  INTERVAL  (every  10  tasks),  and  STOP  times.  Annotate  the  START  time  after  any 
introductory  comments  or  instructions-when  you  begin  voting  on  task  one.  If  your  group  takes  a  break, 
write  down  STOP  and  RESTART  times  (recommend  stopping  at  a  1 0th-task  mark).  Use  HRS/MINS/SECS.. 
Record  each  vote's  consensus  (at  most  3  per  task  per  career  field). 

Based  on  your  observations,  make  any  comments  concerning  anything  which  helped  or  hindered  the 
the  group  process.  Use  the  back  of  the  form,  if  additional  space  is  needed. 

Procedures  for  Completing  Questionnaires 

Have  participants  fill  out  the  Pre-Process  Questionnaire  FIRST  before  any  voting  commences. 

Each  participant  has  a  UNIQUE  GROUP/INDIVIDUAL  NUMBER  that  must  be  used  for  every  questionnaire. 
It  does  not  matter  who  gets  which  number,  except  that  each  individual  uses  the  SAME  number  for  all 
questionnaires  (VERY1MPORTANT). 

Make  sure  the  Environmental  Professional  fills  out  all  questionnaires. 

The  group  can  not  adjourn  each  day  until  after  completing  the  Post-Process  Questionnaire.  Dr  Wolfe’s 
questionnaire  (GDSS  groups  only)  may  be  completed  on  the  bus  ride  back  to  the  Hope  Center  on  FRIDAY. 

General  Instructions 

Be  aware  of  additional  tasks  which  may  have  been  added  by  the  Large  Group  Discussion  and  include 
these  on  the  Data  Collection  Form  and  overhead  acetates. 

The  Environmental  Professional  is  responsible  for  annotating  consensus  levels  and  nonconsensus  levels 
(include  asterisk)  on  the  overhead  acetate. 

Each  day  you  may  not  adjourn  until  two  career  fields  have  been  completely  voted  on  and  the  Post-Process 
Questionnaire  is  completed  by  all  five  participants. 

Figure  3.7.  Instructions  for  Facilitators:  GDSS  and  Facilitated- Manual  Groups. 
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which  may  have  affected  the  process,  either  helping  or  hindering.  All  information  was  recorded  on 
a  Data  Collection  Form9  (reference  Figure  3.8). 

Unfacilitated  manual  groups  were  given  few  instructions  and,  thus,  were  allowed  to  use  what¬ 
ever  voting  techniques  they  desired.  They  could  meet  in  the  large  conference  room  or  elsewhere 
in  the  conference  center.  The  only  stipulations  given  (reference  Figure  3.9)  were  that  they  had  to 
vote  sequentially  (i.e.,  one  career  field  at  a  time,  one  task  at  a  time)  and  the  environmental  repre¬ 
sentative  was  requested  to  annotate  interval  times  on  the  Data  Collection  Form.  The  processes  of 
recording  final  consensus  levels  and  filling  out  questionnaires  were  straightforward. 

All  group  facilitators  and  the  environmental  representatives  (in  manual  groups)  used  the 
following  general  procedures: 

1 .  Welcome  members  and  allow  introductions. 

2.  Administer  pre-process  questionnaires. 

3.  Review  small  group  voting  procedures. 

4.  Vote  on  all  tasks  for  each  career  field,  one  career  field  at  a  time. 

5.  Stop  voting  after  completing  two  career  fields. 

6.  Administer  post-process  questionnaires. 

7.  Remind  participants  to  review  the  schedule  for  the  next  meeting  time  and  location. 

8.  Dismiss  the  participants. 

3.11  Methodology  Taxonomy 

This  research  is  similar  to  several  laboratory  experiments  and  field  studies  presented  in  the 
literature;  however,  it  is  potentially  more  robust  in  design.  The  approach  to  accomplishing  the 
current  research  uses  acceptable  methods  as  noted  by  Vogel  et  al  below: 


Case  studies  provide  an  opportunity  to  evaluate  GDSS  capabilities  when  used  to 
address  complex  questions  in  organizational  settings  with  groups  of  experienced 
decision-makers.  Studies  can  be  longitudinal  as  well  as  single  session,  with  opportu¬ 
nities  to  capture  the  impact  on  project  productivity  and  the  organization.  Accumulated 
case  studies  provide  a  rich  source  of  qualitative  and  quantitative  information  in  the  do¬ 
main  of  applicability  of  GDSS  as  a  function  of  task  and  organizational  characteristics. 
Surveys  can  be  particularly  useful  in  ascertaining  opportunities  for  GDSS  application 
and  penetration  into  corporate  settings.  Field  study  measurements  include  online 
pre-  and  post  session  questionnaires  comparing  the  automated  process  to  the 
manual  process  as  well  as  systematic  recording  of  perceptions...  (42:23-24). 


9Each  career  field  had  its  own  Data  Collection  Form.  The  form  shown  in  Figure  3.8  was  condensed  from  three 
pages  to  one  page  for  conciseness.  The  last  vote,  task  51,  for  each  career  field  was  dropped  to  balance  the  data  into 
20  equal  intervals  across  all  groups. 
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Figure  3.8.  Data  Collection  Form  for  all  Groups. 
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_ instructions  for  Environmental  Professionals _ 

Procedures  for  Group  Process 

Consensus  voting  and  subsequent  discussions  should  be  performed  individually  for  each  task  per 
career  field.  Do  not  vote  for  multiple  tasks  all  at  once. 

Make  sure  that  FIVE  people  vole  -  that  includes  the  Environmental  Professional. 

Procedures  for  Data  Collection 

Be  sure  to  note  GROUP  NUMBER,  CAREER  FIELD,  and  DATE  on  each  Data  Collection  Form  (IMPORTANT). 

Record  START,  INTERVAL  (every  10  tasks  per  career  field),  and  STOP  times.  Annotate  the  ST ART  time  after  any 
introductory  comments  or  instructions-when  you  begin  voting  on  task  one.  If  your  group  takes  a  break,  note  the 
STOP  and  RESTART  times  (recommend  stopping  at  a  lOth-task  mark).  Use  HOURS,  MINUTES,  and  SECONDS. 

Record  consensus/nonconsensus  levels  for  each  task. 

Procedures  for  Completing  Questionnaires 

Have  participants  fill  out  the  Pre-Process  Questionnaire  FIRST  before  any  voting  commence*. 

Each  participant  has  a  UNIQUE  GROUP/INDIVIDUAL  NUMBER  that  must  be  used  'or  every  questionnaire. 

It  does  not  matter  who  gets  which  number,  except  that  each  individual  uses  the  SAME  number  for  all 
questionnaires  (VERY  IMPORTANT). 

Make  sure  the  Environmental  Professional  (that's  you)  also  fills  out  all  questionnaires. 

The  group  can  not  adjourn  each  day  until  after  completing  the  Post-Process  Questionnaire. 

General  Instructions 

Be  aware  of  additional  tasks  which  may  have  been  added  by  the  Large  Group  Discussion  and  annotate  these  on 
the  Data  Collection  Form  and  overhead  acetates. 

The  Environmental  Professional  is  responsible  for  annotating  consensus  levels  and  nonconsensus  levels  (include 
asterisk)  on  the  overhead  acetate. 

Each  day  you  may  not  adjourn  until  two  career  fields  have  been  completely  voted  on  and  the  Post-Process 
Questionnaire  is  completed  by  all  five  participants. 

Figure  3.9.  Instructions  for  Environmental  Professionals — Unfacilitated  Manual  Groups  Only. 
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A  taxonomy  is  a  system  for  classifying  related  materials,  species,  et  cetera  (17:1458).  Al¬ 
though  there  is  no  agreement  on  a  single  methodology  taxonomy  for  GDSS  research,  Vogel  and 
Nunamaker  cite  three  candidate  methodology  taxonomies  in  “Group  Decision  Support  System 
Impact:  MultiMethodological  Exploration.”  First,  they  identify  a  proposed  taxonomy  by  Vo¬ 
gel  and  Wetherbe  (1984)  which  consists  of  Theorem  Proof,  Engineering,  Empirical  (case  study, 
survey,  field  test,  experiment),  and  Subjective/Argumentative.  Second,  they  cite  the  taxonomy 
by  Jenkins  (1985),  the  elements  of  which  are  ordered  in  decreasing  strength  in  hypothesis  test¬ 
ing:  Math  Modeling,  Experimental  Simulation,  Laboratory  Experiment,  Free  Simulation,  Field 
Experiment,  Adaptive  Experiment,  Field  Study,  Group  Feedback  Analysis,  Opinion  Research,  Par¬ 
ticipative  (action)  Research,  Case  Study,  Archival  Research,  and  Philosophical  Research.  Finally, 
they  identify  the  taxonomy  proposed  by  Galliers  and  Land  (1987):  Theorem  Proof,  Laboratory 
Experiment,  Field  Experiment,  Case  Study,  Survey,  Forecasting,  Simulation,  Game/Role  Playing, 
Subjective/Argumentative,  Descriptive/Interpretive,  Action  Research  (42:21-22). 

Based  on  the  previous  discussion,  a  field  experiment  is,  therefore,  a  credible  method  for 
evaluating  GDSS.  Perhaps  not  as  well-controlled  as  the  laboratory  experiment,  the  field  experiment 
is  more  robust  in  that  it  evaluates  impacts  in  real-world  situations.  These  types  of  studies  are 
perhaps  more  meaningful,  since  experimental  excellence  is  meaningless  if  users  fail  to  recognize  the 
usefulness  of  GDSS  in  real-life  situations.  Even  if  GDSS  heips  users  to  produce  outstanding  results, 
they  may  abandon  its  use  should  they  become  dissatisfied  with  either  the  process  or  the  outcomes. 


3.12  Field  Experiment  Design  Characterization 

For  the  benefit  of  other  researchers,  it  is  not  enough  to  state  that  one  is  conducting  field 
research.  Many  parameters  are  needed  to  characterize  or  describe  the  type  of  research  being 
conducted,  so  that  others  may  appreciate  the  specific  focus  and  perhaps  perform  parallel  work, 
follow-on  work,  or  even  a  complete  replication  of  this  experiment.  This  research  is,  therefore, 
identified  in  a  number  of  ways.  The  defining  characteristics  of  this  research  design  are  summarized 
below  and  illustraied  in  Figure  3.10  (9:140-160): 

•  Degree  of  Research  Problem  Crystallization:  This  process  began  with  a  number  of 
hypotheses  for  research,  the  goal  being  to  test  the  hypotheses  relating  to  the  research  sub¬ 
objectives. 

•  Method  of  Data  Collection:  Observations  were  collected  without  solicitation  from  the 
participants  and  participant  interrogation  was  conducted  via  the  use  of  pre-  and  post-process 
questionnaires. 

•  Control  of  Variables:  Variables  were  controlled  and  manipulated  by  research  design.  Some 
variables  could  not  be  controlled,  but  were  measured  if  possible. 
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•  Purpose  of  Study:  The  primary  objective  of  this  research  was  to  observe  interrelationships 
between  the  dependent  and  independent  variables  as  a  result  of  GDSS  or  process  structure 
effects;  however,  some  amount  of  descriptive  study  ref-ulted. 

•  Time  Dimension:  This  research  was  a  “one-shot”  observation;  however,  others  may  use  the 
outcomes  in  a  more  long-term  longitudinal  study. 

•  Topical  Scope:  The  approach  in  this  study  was  to  capture  sample  data,  conduct  statistical 
analyses,  and  draw  conclusions  based  on  the  findings.  The  design  is  primarily  focused  on 
breadth,  not  depth,  although  some  amount  of  case  analysis  is  present. 

•  Research  Environment:  The  research  was  conducted  under  actual  environmental  condi¬ 
tions  using  a  real-world  problem  in  the  actual  workplace. 
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Figure  3.10.  Research  Design  Category  (9:140-160). 


3.13  Experimental  Design 

Keeping  in  mind  the  need  to  account  for  the  structure  added  to  the  small  group  processes, 
the  experimental  design  was  segregated  into  two  parts.  Referencing  Figure  3.11,  Experiment  1  was 
designed  to  measure  GDSS  effects:  comparing  GDSS-supported  groups  (GDSS)  and  non-GDSS 
facilitated  manual  (FAC-MAN)  decision  making  groups.  Experiment  2  was  designed  to  measure 
structure  effects:  comparing  FAC-MAN  and  unfacilitated  manual  (traditional)  decision  making 
groups  The  top  four  blocks  of  each  experiment  show  the  experimental  condition  cross-over  groups 
which  received  both  the  presence  and  absence  of  the  treatment;  whereas,  the  bottom  four  blocks 
of  each  experiment  received  only  one  level  of  a  factor.  Group  4  (in  Experiment  1)  and  Group  7 
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GDSS  Field  Experiment  Design 
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Figure  3.11.  GDSS  Field  Experiment  Design. 


(in  Experiment  2)  were  control  groups.  The  intent  in  building  two  experiments  was  to  separate 
the  effects  due  to  the  GDSS  from  the  potential  effects  due  to  the  structure  added  to  the  decision 
making  process.  The  groups  which  were  exposed  to  a  single  treatment  both  days  were  needed 
to  help  account  for  potential  learning  effects  which  may  have  occurred  in  the  experiments.  This 
precaution  was  taken,  since  some  groups  were  introduced  to  their  small  group  conditions  and  the 
required  task  prior  to  receiving  a  treatment  (GDSS  or  FAC-MAN),  while  others  were  exposed  to 
the  treatment  on  the  first  day  without  prior  small  group  and  task  experience. 

This  field  experiment  was  tailored  from  factorial  experimentation;  however,  there  are  several 
unique  differences  which  did  not  allow  purely  factorial  experimental  analysis.  Ideally,  the  experi¬ 
ment  could  have  been  simplified  by  using  a  22  factorial  design.10  A  22  or  2x2  factorial  design  is 
simpler  to  conduct  (reference  Table  3.3).  It  requires  only  four  runs  and  can  measure  more  than 
one  factor  at  a  time.  In  Table  3.3,  a  “0”  represents  the  lower  level  of  the  factor  or  absence  of 
the  treatment,  and  the  “1”  represents  the  higher  level  of  the  factor  or  presence  of  the  treatment 
(25:454-456),  (2:306-308). 

This  exact  model  was  not  appropriate  for  this  experiment  for  three  reasons.  First,  the  model 
does  not  take  advantage  of  the  availability  of  two  separate  days  for  group  sessions,  but  if  it  had,  it 
would  require  the  addition  of  a  third  factor  “day”  (i.e. ,  another  independent  variable).  Second,  reed- 
world  constraints  on  the  conference  would  not  allow  enough  training  time  for  groups  to  effectively 

10Various  forms  of  factorial  analysis  have  been  popular  in  the  literature  for  the  empirical  studies;  however,  many 
of  these  studies  did  not  use  a  facilitator. 
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Table  3.3.  22  Factorial  Design  (25:456). 


EXPERIMENTAL 

CONDITION 

LEVEL  Or 

A 

NON.QDS8/GD8S 

r  ACT  OR 

B 

UN  PAC’D/ FAC’D 

1 

0 

0 

* 

1 

0 

b 

0 

1 

»b 

X 

1 

use  GDSS  without  a  facilitator  (the  design  for  the  “a”  experimental  condition).  Third,  and  most 
important,  the  2x2  design  does  not  meet  the  objectives  of  this  research  effort — to  study  GDSS 
and  non-GDSS  groups  in  a  similarly  structured  manner  with  comparability  to  other  on-going  field 
studies. 

A  23  (or  2x2x2)  factorial  design  could  have  been  implemented  to  account  for  Day  1  and 
Day  2  as  two  levels  of  an  additional  factor;  however,  real-world  limitations  in  resources  prevented 
this  implementation.  Additionally,  some  GDSS  groups  would  have  needed  to  perform  without  the 
assistance  of  a  facilitator.  Again,  this  approach  would  not  meet  the  research  objectives. 

The  resulting  design  shown  in  Figure  3.11  would  account  for  the  factors  of  interest — either 
presence  or  absence  of  the  GDSS  treatment  effect  (Experiment  1)  and  FAC-MAN  treatment  effect 
(Experiment  2).  Also,  the  selected  design  was  more  robust,  since  it  used  more  participants  (35 
versus  20  for  a  2x2).  Additionally,  this  design  incorporated  the  use  of  control  groups  which  experi¬ 
enced  only  one  level  of  a  given  factor  for  both  days.  Although  simpler  to  perform,  factorial  analysis 
was  inappropriate  for  the  current  experimental  design.  The  precise  statistical  evaluation  methods 
are  described  in  detail  later. 

Evaluating  the  effect  of  structure  was  an  important  consideration  addressed  in  the  design  of 
this  experiment.  An  effective  comparison  of  GDSS  to  non-computer  groups  requires  an  evaluation  of 
similar  processes  in  addition  to  the  processes  of  control  groups.  For  this  reason,  the  experimental 
design  employed  a  manual  process  which  provided  a  similar  level  of  structure,  but  without  the 
automated  .upport.  Structure  was  added  to  the  appropriate  treatments  through  use  of  a  facilitator 
and  simultaneous  voting  using  numbered  index  cards.  This  approach  provided  practically  the  same 
level  of  structure  as  that  of  the  GDSS  groups  which  used  GrovpSystems.  This  approach  also  ensured 
simultaneous  voting  by  the  group  members.  That  is,  members  were  prohibited  from  waiting  to  see 
how  others  voted  before  they  c-st  their  vote.  This  latter  procedure  was  strictly  enforced  throughout 
the  experiment.  The  primary  difference  between  the  GDSS  and  the  FAC-MAN  groups  was  the  lack 
of  a  public  viewing  screen  of  the  Group  Matrix  of  votes  cast  for  the  FAC-MAN  groups.  The 
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traditional  unfacilitated  manual  groups  used  whatever  process  they  deemed  appropriate,  but  there 
were  a  few  restrictions  that  all  groups  had  to  follow: 

•  Only  one  career  field  could  be  considered  at  a  given  time. 

•  Voting  had  to  be  accomplished  on  one  task  at  a  time  (no  working  ahead). 

•  Interval  times  to  show  progress  had  to  be  recorded. 

•  Final  consensus  levels  per  task  had  to  be  recorded. 

3.14  Group  Decision  Support  Variables 

Referring  to  Gray’s  comments  on  the  DeSanctis  and  Gallupe  model  of  group  decision  support 
variables  (introduced  in  Chapter  II),  this  experiment  controlled  or  measured  a  number  of  Input 
Variables  and  Process  Variables,  and  measured  several  Output  Variables.  The  specific  variables 
included  in  this  study  are  indicated  in  Figure  3.12  with  a  small  “black  box”  and  are  discussed 
below. 

3.1 4-1  Input  Variables. 

1.  Group  Factors 

•  Group  History  was  presumed  to  be  negligible;  however,  this  variable  was  measured  us¬ 
ing  a  pre-process  questionnaire  to  determine  how  many  group  members  had  previously 
worked  together. 

•  Group  Size  was  controlled.  Exactly  five  members  were  assigned  to  each  group. 

•  Facilitator’s  Role  was  controlled  through  use  of  standardized  procedures  which  were 
discussed  earlier. 

2.  Individual  Factors 

•  Member  Skills  relating  to  computer  familiarity  were  measured  using  a  pre-process  ques¬ 
tionnaire. 

•  Attitudes  concerning  comfort  with  the  use  of  a  personal  computer  was  measured  using 
a  pre-process  questionnaire. 

3.  Contextual  Factors 

•  Task  Characteristics  were  controlled.  Only  voting  and  discussion  activities  occurred. 

•  Member  Proximity  was  controlled  as  only  face-to-face  meetings. 

•  Time  Pressure  was  loosely  controlled  in  that  each  group  was  allocated  approximately  two 
hours  per  session;  however,  groups  were  instructed  to  take  whatever  time  was  necessary 
to  complete  the  tasks.  Facilitators  used  procedures  which  encouraged  the  voting  process 
to  continue. 

•  GDSS  Training /Support  was  standardized  as  the  facilitator  and  technographer  led  the 
groups  through  a  10-15  minute  training  session  on  voting  procedures  and  assisted  group 
members  whenever  difficulties  or  questions  surfaced. 
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INPUTS 


Group  Factors 


Group  History 
Group  Cohesiveness 
Group  size 
Facilitator's  Role 
Leadership  Type 


Individual  Factors 

Member  Skills 
Attitudes 
Personality  Traits 
Expectations 
Perceived  Status 


Contextual  Factors 

Task  Characteristics 
Member  proximity 
Org’l  Award  Structure 
Time  Pressure 
GDSS  Training/ Support 

Technical  Factors 


HardwardJ Software  Features 
Telecommunications  Technology 
Communications  Media 
System  Configuration 


PROCESS 


Information  Process 

Information  Retrieval 
Information  Sharing 
Information  Uee 


Type  of  Decision  Technique 


Ordinary  Process 
Nominal  Group  Technique 
Delphi 

Brainstorming 
Social  Judgement  Analysis 
Statistical  Aggregation 


Group  Dynamics 


Norm  Development 
Participation 
Communication  Patterns 
Power  Relationships 
Conflict 


OUTPUTS 


Performance  Outcomes 

i 

Decision  Speed 
Decision  Quality 

u 

Secondary  Outcomes 

Extent  of  GDSS  Use 

Attitudes 

■ 

Level  of  Consensus 

Decision  Confidence 

Satisfaction  with  GDSS 

Number  of  Meetings 

Figure  3.12.  Group  Decision  Support  Variables  Included  in  this  Experiment. 
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4.  Technical  Factors 

•  Hardware/Software  Features,  Telecommunications  Technology,  Communications  Media , 
and  System  Configuration  were  controlled.  These  factors  were  discussed  earlier  in  this 
chapter. 

8.14.2  Process  Variables. 

1.  Information  Process 

•  Information  Retrieval,  Sharing,  and  Use  were  all  controlled.  Information  pertaining 
to  environmental  tasks  was  detailed  in  the  large  group  session.  Individuals  took  notes 
and  could  use  these  notes  as  sources  of  information  during  the  small  group  sessions. 
Two  forms  of  automated  (GDSS)  information  sharing,  retrieval,  and  use  were  provided. 
Individuals  had  the  ability  to  recall  the  definitions  for  the  training  levels  (i.e.,  the  voting 
criteria)  and  the  ability  to  recall  previous  votes  in  the  group  matrix. 

2.  Type  of  Decision  Technique 

•  The  experiment  used  attributes  of  both  the  Nominal  Group  Technique  and  Delphi  pro¬ 
cesses.  Members  initially  voted  privately  (NGT  and  Delphi),  the  scores  were  averaged 
(Delphi),  members  discussed  their  voting  rationale  publicly  (NGT),  members  revoted 
privately  (Delphi),  and  the  process  continued  until  consensus  or  the  agreed  upon  three- 
vote  restriction  was  reached,  whichever  came  first. 

3.  Group  Dynamics 

•  Conflict  was  measured  on  post-process  questionnaires. 

3.14-3  Output  Variables. 

1.  Performance  Outcomes 

•  Decision  Speed  was  measured  objectively.  Facilitators  recorded  the  interval  times  to 
complete  20  intervals,  each  consisting  of  ten  decisions.  Each  task  was  completed  either 
when  consensus  was  reached  or  when  three  votes  had  been  taken,  whichever  occurred 
first.  The  researcher  also  decided  to  measure  this  variable  subjectively  by  asking  partic¬ 
ipants  on  a  post-process  questionnaire  how  productive  they  felt  their  group  was  in  terms 
of  time  efficiency. 

•  Level  of  Consensus  and  the  Ability  to  Reach  Consensus  (in  terms  of  the  number  of  votes 
cast)  were  measured  objectively  as  a  performance  outcome.  The  facilitators  recorded 
the  distribution  of  all  votes.  The  researcher  also  measured  this  variable  subjectively  by 
asking  participants  how  productive  they  felt  their  group  was  in  terms  of  ability  to  reach 
consensus. 

2.  Secondary  Outcomes 

•  Altitudes  towards  levels  of  process  difficulty  and  structure,  as  well  as  the  ability  to  stay 
focused  on  the  task  were  measured  using  the  post-process  questionnaire. 

•  Decision  Confidence  was  measured  on  the  post-process  questionnaire  by  asking  the  group 
participants  their  satisfaction  with  the  results  their  group  produced. 

•  Satisfaction  with  GDSS  (and  the  group  process)  was  measured  using  the  post-process 
questionnaire. 
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•  Number  of  Meetings  was  controlled.  This  was  not  a  Secondary  Outcome  for  this  ex¬ 
periment.  It  was  a  Contextual  Factor  limited  by  time  constraints  established  by  the 
conference  sponsors.  Each  group  met  exactly  twice. 


3.15  Questionnaire  Design 

This  research  sought  to  not  only  objectively  measure  performance  outcomes,  such  as  decision 
speed  and  consensus,  but  also  to  measure  through  more  subjective  means  the  perceptions  of  the 
participants.  The  most  logical  approach,  and  one  used  regularly  in  the  GDSS  literature,  was  to  em¬ 
ploy  a  questionnaire.11  The  current  research  intended  to  apply  the  multi- methodological  approach 
espoused  by  Nunamaker,  Zigurs,  and  others  in  the  GDSS  community.  For  example,  Zigurs  states 
that  “judicious  combination  of  multiple  methods  of  research  has  the  greatest  potential. ..a  consen¬ 
sus  is  forming  that  a  multi-methodological  approach  provides  the  greatest  power  of  understanding 
complex  sociotechnological  issues  with  which  GSS  research  deals”  (44:115).  Zigurs  explicitly  rec¬ 
ommends  that  a  combination  of  quantitative  and  qualitative  methods  be  used  within  a  particular 
study.  For  this  study,  a  very  fundamental  problem,  inherent  to  the  entire  Information  Systems 
discipline  (of  which  GDSS  is  a  subset),  could  not  be  overcome:  there  are  no  agreed  upon  means  nor 
validated  measuring  devices  for  collecting  qualitative  data  (i.e. ,  survey  instruments).  Zigurs  states 
the  following  (44:117): 

A  real  need  exists  for  validated  measurement  instruments  in  GSS  research,  and  au¬ 
thors  need  to  provide  more  information  about  measures  they  are  using  to  operationalize 
constructs.  Most  published  studies  reveal  a  dearth  of  information  about  measurement 
instruments  used... these  sample  GSS  articles  are  not  all  unusual  in  their  lack  of  infor¬ 
mation  on  instrument  validation. 

This  problem  of  using  unvalidated  survey  instruments  is  rampant  in  the  entire  Information 
Systems  (IS)  discipline.12  It  was  certainly  not  the  intent  of  this  research  to  develop  and  validate  a 
survey  instrument  for  GDSS  research.  That  would  be  impossible  to  do  under  the  existing  field  study 
conditions;  however,  the  author  recognizes  the  importance  of  using  validated  measuring  devices  in 
experimental  studies. 

The  intent  in  the  development  of  the  current  questionnaire  was  to  use  an  ordinal  scale  similar 
to  the  Likert  7-point  scale  (reference  Figure  3.13)  commonly  used  in  many  research  efforts  and, 
perhaps,  familiar  to  the  prospective  respondents. 

11  The  types  of  questions  asked  were  similar  in  context  to  those  of  Jessup  et  at,  Lewis  et  a],  Nunamaker  et  al,  and 
Gallupe  et  al  (12:286),  (18:270),  (21:200),  (30:190-191). 

12Citing  Straub’s  1989  study,  Zigurs  claims  that  out  of  117  studies,  62%  lacked  a  single  form  of  instrument 
validation.  Also,  Zigurs  cites  Zmud  and  Boyntons'  1989  efforts  which  evaluated  27  recent  IS  articles  and  found  that 
only  three  of  1 1 9  examined  scales  met  the  criteria  for  “internal  consistency,  validity,  and  use  of  multiple  higher-level 
items.”  Zigurs  cites  their  conclusion  that  “IS  survey  instruments  are  at  a  very  early  stage  of  development”  (44:118). 


3-29 


OMgtM  IMMM  AgrM  Agna  Agm 

I']  I I - } 1 1 


Figure  3.13.  Likert  7-point  Ordinal  Scale. 

The  construction  of  questionnaire  scales  similar  to  the  Likert  scale  is  fairly  straightforward, 
since  there  has  already  been  a  great  deal  of  research  in  this  area;  however,  the  scale  still  possesses 
only  ordinal  properties  (39:V-C  Page  4).  This  fact  makes  the  proper  selection  of  response  alter¬ 
natives  critically  important.  The  word  phrases  (i.e.,  the  adjectives  and  adverbs)  chosen  should  be 
clear  and  easy  to  understand.  That  is,  they  should  have  precise  meaning  and  possess  little  possibil¬ 
ity  of  confusing  the  respondents  as  to  whether  one  term  denotes  a  higher  degree  of  “favorableness” 
or  “unfavorableness”  (39:VIII-A,  Page  1).  There  have  been  numerous  studies  which  have  deter¬ 
mined  scale  values  and  variances  for  many  response  alternative  phrases.  The  purpose  of  these  scale 
values  is  to  assist  questionnaire  developers  in  selecting  the  appropriate  response  alternatives — not 
for  assigning  specific  values  to  an  ordinal  scale  for  data  collection  and  analysis  purposes  (39:VII1-A, 
Page  1).  The  response  alternatives  should  have  the  following  properties  (39:VIII-E,  1): 

•  Phrases  with  scale  values  as  far  apart  as  possible. 

•  Phrases  with  scale  values  as  equally  distant  as  possible. 

•  Phrases  with  small  variability  (i.e.,  small  standard  deviations). 

•  Phrases  with  parallel  wording. 

Selection  of  the  scale  phrases  in  this  study  were  extracted  from  sets  of  response  alternatives 
in  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences’  Questionnaire  Con¬ 
struction  Manual.  These  terms  have  been  validated  as  being  one  standard  deviation  apart  and 
have  parallel  wording  (39:VIII-F,  2).  These  were  also  terms  which  the  research  team  felt  were  far 
enough  apart  that  a  potential  respondent  could  accurately  discern  an  interval  and,  thus,  be  able  to 
select  an  absolute  choice. 

The  post-process  questionnaires  (reference  Appendices  A  and  B)  were  identically  constructed 
in  order  to  perform  comparative  statistical  analysis  of  the  responses  associated  with  each  session. 
There  was  one  significant  exception.  The  second  post-process  questionnaire  included  a  section 
which  required  each  respondent  to  compare  one  process  to  the  other  (refer  to  Page  2  of  3,  Question 
9  of  the  post-process  questionnaire  in  Appendix  B).  This  comparison  was  included  for  the  purpose 
of  measuring  the  reliability  of  the  survey  instrument  used. 
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Finally,  three  open-ended  questions  were  included  so  that  participants  could  state  any  pos¬ 
itive,  negative,  or  otherwise  noteworthy  comments  concerning  their  small  group  process.  These 
questions  provided  an  opportunity  to  gain  additional  insight  into  the  results  of  the  group  processes, 
especially  those  which  could  be  impacted  by  group  dynamics. 


3.16  Data  Collection  Techniques 

Although  GroupSystems  has  the  capability  to  record  both  intermittent  and  final  results  of 
the  consensus  voting  in  text  files,  the  process  of  saving  these  files  would  have  used  additional 
time,  thereby  inflating  the  decision  speed  variable.  The  researcher  standardized  the  data  recording 
process  across  all  groups  by  implementing  the  Data  Collection  Form  (Figure  3.8).  The  facilitators 
and  environmental  representatives  recorded  consensus  levels  as  the  small  groups  moved  through 
the  decision  making  process.  The  only  difference  in  the  procedure  was  that  unfacilitated  manual 
control  groups  only  recorded  the  final  vote  for  each  task — these  groups  were  not  constrained  to  any 
particular  number  of  votes;  whereas,  the  GDSS  and  FAC-MAN  groups  recorded  every  vote,  but 
were  limited  to  a  maximum  of  three  votes.  One  limitation  of  this  process  was  that  unfacilitated 
manual  groups  were  limited  to  only  one  of  two  statistical  tests  used  for  evaluating  consensus. 

The  facilitators  and  environmental  representatives  administered  the  pre-process  question¬ 
naires  on  the  first  day  prior  to  any  of  the  decision  making  activities.  The  post-process  question¬ 
naires  were  administered  at  the  end  of  each  group  session  on  both  days.  None  of  the  personnel 
administering  either  the  pre-  or  post-process  survey  instruments  reported  any  difficulties. 


3.1 7  Pretesting 

Emory  states  that  pilot  testing  the  experiment’s  procedures  and  pretesting  data  collection 
instruments,  such  as  questionnaires,  can  potentially  reveal  errors  in  design  and  improper  control 
of  extraneous  or  environmental  conditions,  as  well  as  allow  correction  or  refinement  before  actual 
experimental  testing  (9:422-423).  In  other  words,  this  practice  gives  the  experimenter  one  last 
opportunity  to  look  for  control  problems  which  might  confound  the  results  obtained  in  order  to 
revise  administration  procedures  and  data  collection  methods  before  actual  use. 

For  this  research,  the  experimental  procedures,  operation  of  the  GDSS  software  and  hardware, 
the  data  collection  techniques,  and  the  questionnaires  were  pretested  prior  to  the  beginning  of  the 
conference.  All  the  facilitators,  GDSS  and  FAC-MAN,  participated  in  the  development  of  the  small 
group  voting  and  data  collection  procedures  and  conducted  dry-runs  in  either  a  GDSS  decision  room 
using  GroupSystems  Kor  a  FAC-MAN  conference  room  using  voting  cards.  Armstrong  Laboratory 
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staff  and  technographers  also  participated  to  provide  an  additional  level  of  expertise  and  experience. 
Each  facilitator  rehearsed  his  responsibilities  using  the  actual  tasks  for  the  conference  with  a  mock 
five- member  group.  This  dry  run  provided  facilitators  the  opportunity  to  ask  questions  regarding 
the  procedures  to  be  used  during  the  experiment.  The  facilitators  were  subsequently  critiqued  on 
both  positive  and  negative  aspects  of  the  group  process  procedures  that  were  followed. 

During  these  pretest  practice  sessions,  pre-  and  post-process  questionnaires  were  reviewed  by 
Armstrong  Laboratory  personnel  and  were  administered  to  the  facilitators  and  others  participating 
in  the  exercise.  No  one  experienced  any  difficulty  with  the  word  choices  for  the  ordinal  scales  or 
questions;  however,  several  questions  were  modified  to  improve  clarity  and  standardization. 


3.18  Statistical  Analysis 

The  basic  intent  of  the  statistical  analysis  was  to  determine  whether  groups  which  received 
the  higher  level  of  a  factor — presence  of  GDSS  for  Experiment  1  or  facilitation  for  Experiment  2 — 
exhibited  significantly  different  results  from  those  groups  experiencing  an  absence  of  the  treatment 
(i.e.,  the  lower  level  of  the  factor).  Although  three  statistical  techniques  were  employed  in  this 
study,  one  method  used  for  evaluating  both  performance  and  secondary  outcomes  examined  the 
distributions  of  the  observed  data  to  determine  whether  these  distributions  were  significantly  dif¬ 
ferent.  For  example,  should  the  observations  from  GDSS  groups  result  in  a  significantly  different 
distribution  compared  to  that  of  FAC-MAN  groups,  one  would  conclude  that  the  GDSS  treatment 
had  a  significant  effect  on  the  outcome  variable  being  evaluated.  Based  on  the  results  of  the  sample 
statistical  tests,  one  could  then  make  inferences  about  GDSS  use  in  the  Air  Force  as  a  whole.  The 
process  just  described  was  accomplished  using  hypothesis  testing. 

A  hypothesis  is  a  “relational  statement  describing  an  association  or  dependence  between  two 
or  more  variables”  (9:419).  According  to  Conover,  “the  hypothesis  is  tested  on  the  basis  of  the 
evidence  contained  in  the  sample.  The  hypothesis  is  either  rejected ,  meaning  the  evidence  from  the 
sample  casts  enough  doubt  on  the  hypothesis  for  us  to  say  with  some  degree  of  confidence  that  the 
hypothesis  is  false,  or  accepted ,  meaning  that  it  is  not  rejected”  (4:75).  Typically,  the  hypothesis 
statement  is  actually  worded  in  terms  of  a  null  hypothesis,  Ha,  which  is  usually  intended  to  be 
rejected  as  false  and  an  alternate  hypothesis,  Ha ,  which  is  intended  to  be  accepted  as  true  when  Ha 
is  rejected.  For  this  research,  H0  and  Ha  may  be  uniquely  worded  for  a  specific  dependent  variable 
being  tested.  Several  key  terms  used  in  hypothesis  testing  are  summarized  below  for  conciseness: 

Test  Statistic.  Statistic  compared  to  a  critical  statistic  which  is  used  to  determine  the  decision 

in  hypothesis  testing  (4:77). 
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Critical  Region.  Set  of  random  variables  within  a  sample  space  which  results  in  rejection  of  H0 
(4:78). 

Type  I  Error.  Rejecting  a  true  Ha;  probability  of  making  a  Type  I  error  equals  a  (4:78). 

Level  of  Significance.  Maximum  probability  of  rejecting  a  true  H0\  equals  a  (4:78). 

In  hypothesis  testing,  there  are  essentially  two  classes  of  significance  tests:  parametric  and 
nonparametric.  Parametric  tests  use  interval  or  ratio  measurements  and  are  used  when  information 
about  an  underlying  distribution  is  known.  Nonparametric  tests  use  nominal  (classes)  or  ordinal 
(ordered)  measurements;  however,  they  are  used  when  one  can  not  characterize  the  underlying 
distribution.  Nonparametric  tests  are  statistically  less  powerful  than  parametric  tests,  but  require 
fewer  and  less  stringent  assumptions.13  Although  some  researchers  have  applied  parametric  tests 
for  ordinal  data,  nonparametric  tests  are  the  only  technically  correct  tests  to  use  (9:529-530,  532). 


3.19  Test  Selection  and  Data  Analysis  Procedures 

Emory  states  that  the  researcher  should  determine  the  following  before  selecting  an  appro¬ 
priate  statistical  test  (9:532): 

1.  How  many  samples  are  involved  (1,2,...  Ar)? 

2.  If  there  are  two  or  k  samples,  are  they  independent? 

3.  Is  the  measurement  scale  nominal,  ordinal,  interval,  or  ratio? 

Experiments  (or  sample  sets  of  a  population)  are  independent  if  the  results  of  one  experiment 
(sample)  do  not  influence  the  results  of  another  experiment  (sample)  (4:13).  Conover  and  Emory 
describe  the  measurement  scales  identified  in  item  three  above  as  follows  (4:64-66)  (9:172-176): 

Nominal  Scale.  Observations  are  assigned  numbers  or  names  in  order  to  classify  them  into  dis¬ 
tinct  categories. 

Ordinal  Scale.  Observations  are  assigned  numeric  values  for  ordering  (sizing)  purposes  only.  Dis¬ 
tances  between  numeric  values  may  not  be  fixed. 

Interval  Scale.  Observations  possess  relative  order  and  distance  characteristics  from  other  ob¬ 
servations.  Distances  between  numeric  values  are  equal,  but  the  origin  (or  zero  point)  is 
arbitrarily  chosen. 

Ratio  Scale.  Observations  are  compared  using  a  ratio  measurement  in  addition  to  order  and  size 
characteristics.  This  scale  possesses  an  origin  which  is  considered  as  absolute  zero. 

The  author  acknowledges  that,  historically,  GDSS  researchers  have  frequently  tested  ordinal 
data  using  parametric  tools  in  both  laboratory  and  field  GDSS  studies.  For  example,  GDSS  and 


13  “Nonparametric  tests  often  achieve  95%  the  efficiency  of  parametric  tests.  For  example,  a  nonparametric  test 
using  a  sample  size  of  100  would  have  the  same  statistical  testing  power  as  that  of  a  parametric  test  with  a  sample 
size  of  95"  (9:532). 
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non-GDSS  groups  have  been  compared  using  either  the  t-test  (a  parametric  test)  to  test  average 
(mean)  values  from  questionnaire  responses  (12:289-290),  (13:9),  (21:197).  In  the  current  research, 
the  ordinal  data  was  tested  with  strict  adherence  to  the  more  acceptable  nonparametric  statistical 
methods. 

Decision  speed  observations  were  ratio/interval  data  and  were  analyzed  using  Multiple  Re¬ 
gression  analysis.  Consensus  observations  were  ordinal  data  and  were  analyzed  using  Contingency 
Tables/Chi-Square  test  and  Fisher’s  Exact  test.  Finally,  the  various  dependent  variables  concerned 
with  the  user’s  satisfaction  levels  and  perceptions  in  post-questionnaire  responses  were  ordinal  data 
and  were  analyzed  using  both  Contingency  Tables/Chi-Square  test,  Fisher’s  Exact  test,  and  the 
Sign  test. 

Data  for  the  performance  outcomes,  decision  speed  and  consensus,  were  taken  from  the  inter¬ 
val  data  recorded  by  facilitators  (time  and  consensus)  and  environmental  representatives  (interval 
time  only  for  unfacilitated  manual  groups).  For  convenience,  only  200  of  the  204  intervals  were 
considered.  In  other  words,  the  last  task  for  each  career  field  was  dropped,  since  an  interval  was 
based  on  10  tasks. 


3.20  Analyzing  Decision  Speed  using  Multiple  Regression 

The  researcher  was  interested  in  comparing  GDSS  to  FAC-MAN  (Experiment  1)  and  compar¬ 
ing  FAC-MAN  to  Manual  (Experiment  2)  with  respect  to  decision  speed.  There  were  a  number  of 
independent  variables  associated  with  this  research.  Decision  speed  was  the  dependent  or  response 
variable  which  could  be  related  to  any  one  of  the  independent  variables.  A  popular  method  used 
for  evaluating  such  relationships  is  multiple  regression.  Since  both  experiments  were  divided  into 
20  intervals  of  10  decisions  each,  decision  speed  was  actually  measured  in  terms  of  the  time,  V),  to 
complete  each  of  these  20  intervals. 

A  first  order  model  ofp  —  1  independent  variables  was  chosen  (27:229): 

Yi  —  0o  +  0lXu  +  /?2-^»2  +  '  ■  '  +  0p-l  Xif-l  +  e,  (3-1) 

or  equivalently  (27:230): 


Yi 


p-i 

=  00  +  0kXik  +  €i 


*  =  1 


(3.2) 
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where  0O  is  the  intercept,  fa,..  -,0p-i  are  the  regression  coefficients  for  each  of  the  independent 
variables,  Xu,  c,  is  the  error  term,  and  p  is  the  number  of  independent  variables  including  the 
intercept. 

Since  many  of  the  independent  variables  in  this  research  were  not  quantitative  (i.e. ,  not 
numeric),  indicator  or  dummy  variables  were  chosen  in  their  place  (11:146).  Consider  Experiment  1, 
for  example.  The  indicator  variables  helped  to  test  the  model  by  representing  the  absence  of  a 
treatment  (e.g.,  FAC-MAN)  or  initial  experimental  condition  (e.g.,  Day  1)  with  a  “0”  and  the 
presence  of  a  treatment  (e.g.,  GDSS)  or  second  experimental  condition  (e.g.,  Day  2)  with  a  “1.” 
This  test  was  performed  to  determine  if  groups  which  used  GDSS  reached  decisions  faster  than 
groups  which  did  not  use  GDSS.  The  same  procedure  was  followed  for  Experiment  2. 

The  model  includes  several  variables  which  may  account  for  the  variability  in  decision  speed. 
These  variables  address  the  differences  between  groups,  differences  between  days,  and  learning 
effects  associated  with  differences  in  intervals.  Specifically,  the  model  and  the  variables  assigned  to 
the  X i  are  defined  below: 

For  Experiment  1: 

Y  —  fa  +  faXi  +  faX?  4-  fa  A3  +  0^X4  +  0$X$  +  0qXq  +  e 

A!  =  Group  1,  where  Ai  =  1  when  Group  1  is  present,  else  X\  —  0. 

A 2  =  Group  2,  where  A2  =  1  when  Group  2  is  present,  else  A2  =  0. 

A3  =  Group  3,  where  A3  =  1  when  Group  3  is  present,  else  A3  =  0. 

X4  =  Day,  where  A4  =  0  for  Day  1  and  A4  =  1  for  Day  2. 

A5  =  Method,  where  A5  =  0  for  FAC-MAN  and  A5  =  1  for  GDSS. 

Xq  =  Interval  number  (integer  from  1  to  20). 

Groups  1-3  are  compared  to  the  control  group,  Group  4,  which  is  included  in  the  model,  but  not 
represented  as  an  indicator  variable  (i.e.,  the  experimental  condition  where  A1;  A2,  and  A3  all 
equal  zero). 

For  Experiment  2: 

V  =  0o  +  0i  A 1  +  02X3  +  03X3  +  0^X4  +  03  Xs  •+-  0eXe  4-  e 

At  =  Group  4,  where  Ai  =  1  when  Group  4  is  present,  else  Ai  =  0. 

A'2  =  Group  5,  where  A2  =  1  when  Group  5  is  present,  else  A2  =  0. 
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X3  =  Group  6,  where  X$  =  1  when  Group  6  is  present,  else  X3  =  0. 

X4  =  Day,  where  X4  =  0  for  Day  1  and  X4  =  1  for  Day  2. 

X$  =  Method,  where  X$  =  0  for  Manual  and  X5  =  1  for  FAC-MAN. 

X$  =  Interval  number  (integer  from  1  to  20). 

Groups  4-6  are  compared  to  the  control  group,  Group  7,  which  is  included  in  the  calculations, 
but  not  represented  as  an  indicator  variable  (i.e.,  the  experimental  condition  where  Xj,  X3,  and 
X3  all  equal  zero). 

The  hypothesis  for  Experiment  1: 

H0:  There  is  not  a  Statistically  Significant  Relationship  between  the  Decision  Method  (GDSS) 
and  Decision  Speed,  that  is  =  0. 

Ha:  There  is  a  Statistically  Significant  Relationship  between  the  Decision  Method  (GDSS) 
and  Decision  Speed,  that  is  /?5  /  0. 

The  hypothesis  for  Experiment  2: 

H„ :  There  is  not  a  Statistically  Significant  Relationship  between  the  Decision  Method  (FAC- 
MAN)  and  Decision  Speed,  that  is  /?5  =  0. 

//„:  There  is  a  Statistically  Significant  Relationship  between  the  Decision  Method  (FAC- 
MAN)  and  Decision  Speed,  that  is  /?5  /  0. 

The  Test  Statistic  was  the  t-ratio,  T,  where  a  =  .05.  This  statistic  is  used  to  test  the  significance 
of  the  individual  parameter  estimates,  most  importantly  /? 5  (11:23). 

The  decision  concerning  the  test  statistic  was  to  reject  H„  when  T  >  ta/ 2i„,  where  a  =  .05, 
v  =  n  -  m  -  1  degrees  of  freedom,  v  is  the  degrees  of  freedom,  n  is  the  number  of  observed  interval 
times,  and  m  is  the  number  of  independent  variables  excluding  the  intercept.  Equivalently,  one 
would  reject  H0  when  p  <  a,  where  a  =  05. 

This  analysis  was  performed  using  PROC  REG  in  the  statistics  program  SAS.  A  prerequi¬ 
site  for  regression  analysis  is  that  the  observed  data  being  evaluated  are  either  ratio  or  interval 
(36:313).  Additionally,  diagnostic  tests  were  performed  to  ensure  that  the  model  did  not  violate 
the  assumptions  of  regression  and  to  check  for  multicollinearity. 

One  such  assumption  is  that  the  error  terms,  or  residuals,  are  assumed  to  be  independent  and 
randomly  distributed.  In  other  words,  there  should  be  no  correlation  (or  autocorrelation)  of  the 
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error  terms,  e,-.  Freund  describes  this  phenomenon  as  errors  at  time  t  relating  to  previous  errors, 
say  time  t  —  1.  He  states  that  if  autocorrelation  is  ignored,  the  regression  coefficients  are  subject  to 
unknown  biases  (11:68).  Consequently,  the  Durbin- Watson  D  statistic  was  calculated  to  determine 
whether  or  not  the  error  terms  were  correlated.  For  this  test,  the  hypotheses  were  H0  ■  p  —  0  and 
Ha  :  P  >  0,  where  p  is  the  autocorrelation  parameter  (27:450-451).  D  values  were  compared  to 
the  upper  (do)  and  low«r  (di)  bounds  for  this  statistic  in  Table  A-6  of  Applied  Linear  Regression 
Models  by  Neter  et  al  at  sample  size  n,  a  =  .05,  and  the  number  of  independent  variables,  m.M  If 
the  test  resulted  in  D  >  •*  r,  the  test  concluded  H0\  D  <  d^,  Ha\  and  dt  <  D  <  do,  inconclusive 
(27:451).  As  a  general  rule,  if  the  values  of  D  are  close  to  2,  then  the  errors  are  uncorrelated 
(35:1434).  The  randomness  of  the  error  terms  were  further  evaluated  by  examining  the  randomness 
of  residuals  on  a  residual-predicted  value  plot. 

Another  assumption  of  regression  analysis,  the  normality  of  the  distribution  of  the  residuals, 
was  tested  by  implementing  the  Shapiro- Wilk  test  in  SAS.  If  the  correlation  coefficient,  W ,  of  the 
Shapiro-Wilk  test  was  greater  than  .9,  the  residuals  were  concluded  to  be  normally  distributed 
(27:120).  This  conclusion  was  also  supported  by  examining  the  residuals  on  both  Stem-Leaf  and 
Normality  Probability  plots  generated  using  SAS. 

Outliers  are  observations  which  do  not  appear  to  fit  the  model,  and  may  affect  the  accuracy  of 
parameter  estimates.  Although  they  should  not  be  eliminated  from  the  model  unless  the  analysis 
reveals  that  the  observations  were  erroneous,  one  should  attempt  to  account  for  their  presence. 
Outliers  can  be  readily  identified  using  studentized  residuals.  Studentized  residuals  are  the  residuals 
divided  by  their  standard  errors.  Since  values  exceeding  2.5  are  rare  in  the  t  distribution,  studentized 
residuals  exceeding  the  absolute  value  of  2.5  should  be  considered  as  outliers  (11:48-49). 

Finally,  the  model  was  tested  to  ensure  constancy,  or  equality,  of  the  residual  variance.  This 
assumption  was  tested  by  ensuring  that  the  pattern  of  residuals  about  predicted  values  of  the  de¬ 
pendent  variable  on  a  residual-predicted  value  plot  was  fairly  random  in  a  horizontal  band  about 
the  prediction  line  (27:114).  Also,  fit  of  the  model  and  the  possibility  of  omitting  critical  indepen¬ 
dent  variables  were  evaluated  by  checking  the  model’s  F  and  p  statistics  and  examining  various 
residual  plots,  respectively. 

Since  there  were  a  large  number  of  independent  variables  in  the  model,  a  test  for  multi- 
collinearity  was  also  p  mied.  Multicollinearity  means  that  there  is  a  high  degree  of  correlation 
between  the  independent  variables,  A,  .  Multicollinearity  occurs  where  the  independent  variables 

14 Table  A-6  uses  p  -  1  as  the  number  of  independent  variables.  This  nomenclature  was  modified  here  to  preclude 
confusion  with  p,  the  probability  of  rejecting  a  true  H0. 
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measure  similar  phenomena  (11:75).  Freund  states  that  multicollinearity  does  not  violate  the  re¬ 
gression  assumptions,  but  it  may  “inhibit  the  usefulness  of  the  results”  (11:75).  He  cites  three 
specific  outcomes  of  multicollinearity  (11:75-76): 

1.  No  effect  to  the  estimation  of  the  dependent  variable — The  Y  values  are  the  best  unbiased 
estimates. 

2.  Tends  to  inflate  variances  of  the  predicted  values  for  X  values  not  included  in  the  sample. 

3.  Tends  to  inflate  the  variances  of  the  parameter  estimates.  The  regression  coefficients  could 
also  have  incorrect  signs  or  not  be  judged  as  statistically  significant. 

Variance  Inflation  Factors  (VIF)  were  calculated  for  all  the  independent  variables.  These 
numbers  were  compared  to  the  model  VIF=  1/(1  —  Rf),  where  Ri  is  the  coefficient  of  determination 
of  the  regression  of  the  ith  independent  variable  on  all  other  independent  variables  (11:80).  Any 
variables  associated  with  VIF  greater  than  that  of  the  model  VIF  were  considered  to  be  more  closely 
associated  with  the  other  independent  variables  than  with  the  dependent  variable.  An  arbitrary 
V'lF  level  of  10  was  chosen  as  a  level  of  concern  (11:80). 

3.21  Analyzing  Consensus  using  Contingency  Tables/Chi-Square  Test. 

The  objective  was  to  determine  if  use  of  a  GDSS  or  the  presence  of  structure  affected  the 
variable  consensus  in  a  small  group  decision  making  process.  If  so,  one  would  expect  to  see  statisti¬ 
cally  significant  differences  in  the  distributions  of  observations  taken  from  the  GDSS,  FAC-MAN,  or 
Manual  processes.  Two  statistical  tests  were  run  for  consensus:  1)  the  ability  to  reach  consensus  in 
terms  of  the  number  of  votes  it  took  to  obtain  consensus  for  each  task  and  2)  the  level  of  consensus 
in  terms  of  the  number  of  the  total  tasks  for  which  groups  actually  reached  consensus. 

The  first  test,  ability  to  reach  consensus,  could  only  be  performed  for  Experiment  1.  A  similar 
analysis  between  FAC-MAN  and  Manual  groups  could  not  be  conducted  due  to  the  lack  of  data. 
Since  the  experiments  were  designed  primarily  to  measure  decision  speed  without  placing  undue 
constraints  on  the  Manual  experimental  condition,  only  a  final  consensus  level  was  recorded  by  the 
Manual  groups.  Otherwise,  such  restrictions  could  have  potentially  confounded  the  speed-related 
statistical  results. 

During  the  experiment,  all  facilitators  recorded  the  number  and  distribution  of  votes  cast 
for  individual  environmental  tasks  in  order  to  perform  these  statistical  evaluations.  The  following 
assumptions  for  using  contingency  tables  were  met:  ’he  outcomes  of  the  observations  were  mu¬ 
tually  independent,  and  each  observation  was  categorized  into  exactly  one  class  or  column  of  the 
contingency  table  (4.154). 
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Figure  3.14.  Contingency  Table  for  Consensus  Voting. 


For  the  first  test,  a  contingency  table  similar  to  that  in  Figure  3.14  was  constructed  for  GDSS 
and  FAC-MAN  groups.  Observations  from  each  task  of  the  voting  process  were  later  separated 
into  distinct  classes  according  to  the  number  of  votes  required  to  reach  consensus:  “1”,  “2,”  “3,” 
or  “Never”  (i.e.,  non-consensus). 


The  hypothesis  for  the  first  consensus  test,  Experiment  1  only,  is  provided  below  (4:154). 

H0 :  There  is  No  Difference  between  GDSS  and  FAC-MAN  in  Relationship  to  the  Ability  to 
Reach  Consensus,  that  is  all  probabilities  in  the  same  column  are  equal  to  each  other  (pi;  =  pij  — 
Pij  for  all  j  =  1,2,...,  c). 

Ha’.  There  is  a  Difference  between  GDSS  and  FAC-MAN  in  Relationship  to  the  Ability  to 
Reach  Consensus,  that  is  at  least  two  probabilities  in  the  same  column  are  not  equal  to  each  other 
( pij  ^  pkj  for  some  j  and  some  pair  i  and  k). 

For  GDSS,  FAC-MAN,  and  Manual  processes,  the  overall  level  of  consensus  was  analyzed 
using  contingency  tables  and  the  Chi-Square  test.  Here,  the  final  consensus  recorded  for  each  task 
was  analyzed  for  both  experiments.  The  contingency  table  was  constructed  with  observations  of  the 
task  votes  being  placed  into  two  classes,  “Yes”  and  “No,”  which  signified  whether  or  not  consensus 
was  reached. 


The  hypotheses  for  the  second  consensus  test,  level  of  consensus,  are  provided  below  (4:154). 

For  Experiment  1: 

H0  There  is  No  Difference  between  GDSS  and  FAC-MAN  in  Relationship  to  the  Level  of 
Consensus,  that  is  all  probabilities  in  the  same  column  are  equal  to  each  other  (pij  =  p^j  =  Pij  for 
all  j  =  1,2, . .  ,,c). 
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Ha:  There  is  a  Difference  between  GDSS  and  FAC-MAN  in  Relationship  to  the  Level  of 
Consensus,  that  is  at  least  two  probabilities  in  the  same  column  are  not  equal  to  each  other 
( Pij  ^  pkj  for  some  j  and  some  pair  i  and  k). 


For  Experiment  2: 

H0:  There  is  No  Difference  between  FAC-MAN  and  Manual  in  Relationship  to  the  Level  of 
Consensus,  that  is  all  probabilities  in  the  same  column  are  equal  to  each  other  (pij  =  p-Z]  =  ptJ  for 
all  j  =  1,2, . .  .,c). 

Ha:  There  is  a  Difference  between  FAC-MAN  and  Manual  in  Relationship  to  the  Level 
of  Consensus,  that  is  at  least  two  probabilities  in  the  same  column  are  not  equal  to  each  other 
(pij  ^  pkj  for  some  j  and  some  pair  i  and  k). 

The  Test  Statistic  for  analyzing  r  x  c  tables  is  (25:292): 


*’  =  ££ 

<=i  j= i 

which  can  be  shown  to  be  equivalent  to  (4:159): 


(Oq  ~  Eijf 

Ea 


(3.3) 


r  c  O? 
.  =  1,7  =  1  13 


(3.4) 


where  OtJ  is  the  observed  cell  frequency,  Eij  is  the  expected  cell  frequency,  and  N  is  the  total 
number  of  observations  for  all  samples. 

The  following  computations  are  required  to  compute  the  formulae  in  Equations  3.3  and  3.4 
(4:154-155): 


N  =  ni  +  n2  4- - 1-  rij 

^  _tiiX  Cj 

Eij  ~  N 

Cj  =  Oij  +  Oij  +  •••-(-  Oij  forj  =  1, 2, . . . ,  c 


Finally,  Equation  3.4  is  reduced  to  the  following  equation  for  2x2  tables,  such  as  that  used 
for  test  1,  ability  to  reach  consensus  (4:155): 
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Figure  3.15.  Contingency  Table  for  Questionnaire  Responses. 
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The  decision  concerning  the  test  statistic  was  to  reject  H0  when  \2  >  X2  Vi  where  a  =  .05 
and  v  —  (r  —  l)(c  —  1)  degrees  of  freedom  or,  equivalently,  Reject  H0  when  p  <  a,  where  a  =  .05. 


3.22  Analyzing  Participant  Responses  using  Contingency  Table/Chi-Square  Test 

Two  sets  of  statistical  tests  were  chosen  to  evaluate  the  statistical  significance  of  participant’s 
responses  on  the  post-process  questionnaires.  The  first  test  was  the  contingency  table/Chi-Square 
test  described  earlier.  For  this  particular  test,  one  additional  consideration  is  worthy  of  comment. 
For  ordinal  values  one  through  seven  on  the  questionnaire,  several  observed  and  expected  frequencies 
were  low  or  even  zero.  According  to  Schlotzhauer  and  Littell  of  SAS,  Inc.,  statisticians  disagree  on 
whether  to  trust  the  results  of  the  Chi-Square  test  whenever  a  cell’s  expected  frequency,  Eij,  falls 
below  five  (5).  They  recommend  following  the  Chi-Square  test  with  the  more  conservative  Fisher’s 
Exact  Test15,  which  is  based  on  a  hypergeometric  distribution,  to  be  sure  the  results  from  the 
Chi-Square  test  are  correct  (36:371).  Mehta  and  Patel  state  “for  sparse  contingency  tables,  fairly 
large  differences  can  exist  between  the  p  values  generated  by  Fisher’s  Exact  test  and  by  Pearson’s 
X2  test.  Inferences  based  on  the  x2  test  might,  therefore,  be  misleading,  whereas  absolute  reliance 
can  be  placed  on  the  significance  level  generated  by  Fisher’s  Exact  test”  (24:432).  The  Fisher’s 
Exact  test  was  run  in  addition  to  every  Chi-Square  test  to  ensure  the  accuracy  of  the  p  values. 

15  Fisher  developed  an  exact  method  for  evaluating  2x2  contingency  tables  with  expected  frequencies  too  small  for 
the  x2  results  to  be  fully  trusted.  This  approach  is  provided  by  D.  J.  Finney  in  Biomctrika ,  Vol  35,  pp  145-156, 
1948.  This  method  has  been  expanded  for  use  beyond  a  2x2  contingency  table  in  SAS\  however,  further  discussion 
of  this  test  is  outside  the  scope  of  this  research. 
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The  contingency  table/Chi-Square  test  procedures  are  identical  to  those  of  the  objective 
measures  of  effect  (i.e.,  the  consensus  tests)  with  a  contingency  table  similar  to  that  shown  in 
Figure  3.15.  The  numbers,  1-7,  in  this  figure  indicate  the  classes  of  response  alternatives  given 
on  the  questionnaire’s  ordinal  scale  (reference  Appendix  A  and  B  for  the  exact  wording  of  the 
response  alternatives).  The  hypotheses  are  similar  to  those  in  the  consensus  tests,  except  that  they 
now  reflect  evaluation  of  the  distribution  of  responses  obtained  from  the  subjective  data  obtained 
from  the  post-process  questionnaires. 

For  Experiment  1: 

H0:  There  is  No  Difference  between  GDSS  and  FAC-MAN  in  Relationship  to  the  Dependent 
Variable  of  Interest. 

Ha:  There  is  a  Difference  between  GDSS  and  FAC-MAN  in  Relationship  to  the  Dependent 
Variable  of  Interest. 

For  Experiment  2: 

Ha:  There  is  No  Difference  between  FAC-MAN  and  Manual  in  Relationship  to  the  Dependent 
Variable  of  Interest. 

Ha:  There  is  a  Difference  between  FAC-MAN  and  Manual  in  Relationship  to  the  Dependent 
Variable  of  Interest. 

Here,  the  dependent  variable  of  interest  equates  to  the  responses  measured  on  the  post-process 
questionnaires,  namely:  satisfaction  with  the  results,  satisfaction  with  the  process,  level  of  difficulty, 
ability  to  remain  focused,  productivity  in  terms  of  time  efficiency,  productivity  in  terms  of  the  level 
of  consensus,  level  of  conflict,  ability  to  reach  consensus,  and  level  of  structure.  The  test  statistic 
and  decision  rule  remain  the  same  for  each  of  these  tests. 

3.23  Analyzing  Participant  Responses  using  Sign  Test 

As  Conover  mentions,  the  Sign  Test  is  the  oldest  of  nonparametric  tests  (circa  1710),  and  is 
based  on  a  binomial  test  with  probability  of  one-half,  p=  \ .  The  Sign  Test  is  useful  for  testing 
whether  one  random  variable  in  a  bivariate  random  sample  (a  pair  of  observations)  tends  to  be 
larger  than  another  and  is  especially  useful  for  testing  for  trends  in  a  series  of  ordinal  measurements 
(4:122).  The  Sign  Test  has  good  power  efficiency  for  small  samples  which  is  particularly  important 
in  this  research  (25:571).  The  data  being  tested  were  collected  in  bivariate  pairs  (A,,  Y,)  which  met 
the  following  assumptions  (4:123): 
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1.  The  bivariate  random  variable  (X<,yi)  where  *  =  1,2,  ...,n  were  mutually  independent. 

2.  The  measurement  scale  was  at  least  ordinal  within  each  pair. 

3.  The  bivariate  pairs  were  internally  consistent,  in  that  if  p(+)  >  p(— )  for  one  pair  (X,-,  Y|), 

then  p(+)  >  p(— )  for  all  pairs  and  vice-versa. 

In  addition  to  the  contingency  table/Chi-Square  test,  the  researcher  had  the  opportunity 
to  directly  compare  groups  which  experienced  both  experimental  conditions  using  the  Sign  test. 
Instead  of  classifying,  counting,  and  looking  at  the  overall  distribution  of  responses  for  individual 
days,  here  matched  pairs  of  responses  between  the  two  experimental  conditions  were  examined. 
This  procedure  differs  from  the  previous  test  in  that  it  considers  only  those  groups  (1,  2,  5,  6) 
which  crossed  treatments  in  one  of  the  two  experiments  (e.g.,  GDSS  on  one  day  and  FAC-MAN 
on  the  other  for  Experiment  1).  All  higher  level  treatment  responses  (GDSS  for  Experiment  1 
and  FAC-MAN  for  Experiment  2)  for  each  questionnaire  question  (i.e.,  each  dependent  variable) 
were  compared  individually  to  the  lower  level  of  the  treatment  (FAC-MAN  for  Experiment  1  and 
Manual  for  Experiment  2)  responses  to  determine  which  level  had  the  higher  response.  If  GDSS 
for  Experiment  1  or  FAC-MAN  for  Experiment  2  were  rated  higher,  a  “+”  was  assigned  for  this 
matched  pair.  If  the  opposite  were  true,  a  was  assigned.  Finally,  if  the  two  responses  were 
equal,  a  “0”  was  assigned  and  the  total  count,  n,  was  decremented  by  one.  The  total  number  of 
pluses  and  minuses  were  summed  to  yield  n.  The  total  number  of  pluses,  T,  was  compared  to  a 
binomial  distribution  at  the  value  of  n  with  p  =  .50  to  determine  if  T  was  large  enough  to  reject 
H0  at  a  =  .05.  The  null  and  alternate  hypotheses  were  as  follows: 

For  Experiment  1: 

H0  :  p(+)  <  p(— )  or  GDSS  <  FAC-MAN  for  the  Dependent  Variable  of  Interest. 

Ha  '■  p(+)  >  p(— )  or  GDSS  >  FAC-MAN  for  the  Dependent  Variable  of  Interest. 

For  Experiment  2: 

H0  :  p(+)  <  p(-)  or  FAC-MAN  <  Manual  for  the  Dependent  Variable  of  Interest. 

Ha  ■  p(+)  >  p(~)  or  FAC-MAN  >  Manual  for  the  Dependent  Variable  of  Interest. 

Again,  the  dependent  variable  of  interest  represents  the  responses  measured  on  the  post¬ 
process  questionnaires:  satisfaction  with  the  results,  satisfaction  with  the  process,  level  of  difficulty, 
ability  to  remain  focused,  productivity  in  terms  of  time  efficiency,  productivity  in  terms  of  level  of 
consensus,  level  of  conflict,  ability  to  reach  consensus,  and  level  of  structure. 
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3.24  Determining  Questionnaire  Reliability  using  Sign  Test 

As  was  mentioned  earlier,  there  exist  few  published,  much  less  validated,  measurement  in¬ 
struments  in  the  GDSS  research.  As  a  tertiary  objective,  this  research  sought  to  demonstrate  the 
reliability  of  the  post-process  questionnaires  by  including  a  comparison  section  on  the  post-process 
questionnaire  for  the  second  day  (reference  Appendix  B,  Questionnaire  page  2-3).  This  section 
asked  the  participant  to  evaluate  each  dependent  variable  as  being  “higher  today”,  “higher  yes¬ 
terday,”  or  “unchanged.”  In  this  manner,  this  data  could  be  compared  against  the  actual  ordinal 
ratings  given  for  treatment  exposures  on  both  days.  The  null  and  alternate  hypotheses  were  as 
follows: 

H0  :  p(+)  <  p(— )  or  Number  of  Times  Objective  and  Subjective  Results  Agree  <  Number  of 
Times  Objective  and  Subjective  Results  Disagree. 

Ha  :  p(+)  >  p(— )  or  Number  of  Times  Objective  and  Subjective  Results  Agree  >  Number  of 
Times  Objective  and  Subjective  Results  Disagree. 

As  before,  this  procedure  was  reserved  for  only  the  groups  which  crossed  treatments.  The 
analysis  was  performed  identically  to  that  mentioned  above,  except  here  an  a  =  .10  was  chosen.  The 
a  value  was  relaxed 
to  the  results  of  this 


somewhat,  since  the  risk  associated  with  a  Type  I  error  was  not  as  significant 
study  compared  to  that  of  the  performance  and  secondary  outcome  objectives. 
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IV.  Results 


This  chapter  addresses  in  detail  the  results  of  the  data  collection  efforts  and  subsequent 
statistical  evaluations  conducted  in  this  study.  First,  specific  insights  into  the  demographics  of  the 
participants,  at  the  group  level  and  as  a  whole,  are  addressed.  Second,  a  brief  discussion  of  several 
complications  experienced  during  the  actual  experimentation  and  their  ramifications  to  this  study 
is  provided.  Finally,  the  results  of  the  statistical  tests  are  discussed,  along  with  several  additional 
findings. 

4-1  Group  Participant  Demographics 

The  researcher  sought  to  characterize  the  composition  of  the  groups  by  measuring  specific 
aspects  of  each  participant’s  background  on  a  pre-process  questionnaire.  Appendix  C  provides  a 
comprehensive  listing  of  the  questionnaire  responses  in  tabular  form.  These  responses  are  summa¬ 
rized  below  for  comparison  purposes. 

4-1.1  Backgrounds  of  Group  Participants.  Rank,  age,  and  years  within  the  career  field  or 
present  organization  tend  to  show  the  organizational  seniority  and  the  management  level  of  the 
participants.  Based  on  the  questionnaire  results,  the  participants  included  in  this  study  worked 
in  primarily  the  middle  management  arena.  Five  of  the  35  participants  did  not  disclose  their 
grade/rank,  and  two  did  not  identify  their  gender. 

Referencing  Figure  4.1,  the  average  age  was  42  years.  The  youngest  participant  was  32;  the 
oldest  was  60.  Group  4  was  the  youngest  group,  averaging  38.6  years;  Group  2  was  the  oldest 
averaging  48.4  years. 

The  median  rank  across  all  groups  was  “Major/GS-13”  (reference  Table  4.1);  the  mode  was 
also  Major/GS-13.  The  lowest  rank  in  the  study  was  a  Technical  Sergeant.  The  highest  ranking 
participants  were  a  GS-15  and  a  Colonel.  Group  2  had  the  highest  ranking  group  with  three 
Lt  Colonels/GS-14s.  The  lowest  ranking  group  was  Group  61  with  a  median  and  modal  rank  of 
Captain  or  GS-11/12.  Overall,  these  levels  of  rank  are  fairly  typical  of  middle  management  within 
the  USAF. 

Years  in  both  the  career  field  and  the  present  organization  are  also  indicators  of  the  level 
of  management  and  the  level  of  experience.  An  important  fact  to  consider  is  that  military  mem¬ 
bers  tend  to  move  every  three  to  four  years.  Consequently,  the  number  of  years  in  the  current 
organization  may  not  represent  experience  as  accurately  as  the  number  of  years  in  a  career  field. 

’Two  participants  in  this  group  did  not  disclose  their  rank. 
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Table  4.1.  Comparison  of  Grade/Rank  Across  Groups. 
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Figure  4.2.  Comparison  of  Years  of  Experience  Across  Groups. 


Referencing  Figure  4.2,  the  average  number  of  years  in  the  career  field  and  the  current  organization 
was  13.54  years  and  4.45  years,  respectively.  Focusing  on  career  field  experience,  Group  5  had  the 
highest  experience  level  with  16.8  years;  Group  7,  the  lowest  with  7.8  years.  Except  for  Group  7, 
all  groups  averaged  over  a  decade  of  experience  in  their  particular  line  of  work. 

Levels  of  education  for  each  of  the  groups  are  shown  in  Table  4.2.  Most  groups  included 
members  who  had  completed  graduate-level  work.  The  overall  sample  median  and  mode  were  both 
at  the  graduate  degree  level.  Five  of  the  thirty-five  participants  (14%)  had  a  PhD  or  MD.  Nineteen 
of  thirty-five  (54%)  had  a  graduate  degree  or  better,  and  only  three  individuals  had  not  completed 
their  undergraduate  degree  (8.6%).  Group  1  stands  out  as  possibly  the  most  educated  group  with 
four  graduate  degrees  and  one  PhD/MD.  Groups  2  and  6  had  the  least  educated  participants. 


Table  4.2.  Comparison  of  Education  Across  Groups. 


EDUCATION  LEVEL 

INDEX 

GP  1 

GP  3 

GP  3 

GP  4 

GP  5 

GP  6 

GP  T 

High  School  Diploma 

1 

0 

0 

0 

0 

0 

0 

0 

College  Courses  Taken 

2 

0 

0 

1 

0 

0 

1 

1 

Undergraduate  Degree 

3 

0 

1 

0 

1 

1 

0 

0 

Graduate  Courses  Taken 

4 

0 

3 

1 

2 

1 

2 

1 

Graduate  DegTee 

5 

4 

1 

3 

1 

3 

1 

1 

PhD  or  MD 

6 

1 

0 

0 

1 

0 

1 

2 

SAMPLE  MEDIAN 

5 

A 

5 

4 

5 

4 

S 

SAMPLE  MODE 

5 

4 

5 

4 

5 

4 

6 

As  far  as  gender  is  concerned,  at  least  26  out  of  the  35  respondents  (77%  of  the  participants) 
were  male.2  Two  participants  did  not  identify  their  gender.  Two  of  the  three  GDSS  groups,  Groups 
1  and  3,  had  no  females,  nor  did  Group  5.  Group  2  had  one  female,  and  Groups  4,  6,  and  7  all  had 
two  females.  Experiment  2  included  twice  as  many  females  as  did  Experiment  1 . 

Finally,  it  was  important  to  determine  whether  or  not  group  members  knew  each  other.  If 
there  were  groups  which  had  previously  worked  together,  group  cohesion  based  on  previous  work 
experience  might  have  confounded  the  data.  On  average,  the  participants  knew  one  other  member 
of  their  group.  Only  one  person  knew  three  members  in  her  group  (Group  7),  and  13  participants 
(37%)  did  not  know  anyone  in  their  group. 

With  the  exception  of  gender,  all  the  groups  appeared  evenly  balanced  with  respect  to  the 
previously  mentioned  demographic  variables.  The  groups  appeared  fairly  experienced  in  terms  of 
education  and  time  in  the  career  field.  In  terms  of  age  and  rank,  the  participants  came  primarily 
from  middle  management.  Group  7  was  the  exception  in  two  respects.  First,  this  group  had  the 
lowest  experience  level  in  the  career  field  with  7.8  years — almost  half  the  overall  average  of  13.5 
years.  Second,  the  members  in  Group  7  knew  more  members  in  their  group  than  any  other  group. 
For  example,  one  member  knew  three  of  the  other  four  members,  and  two  members  knew  two  of  the 
other  four  members.  Each  member  in  this  group  knew  on  average  twice  as  many  group  members 
as  did  other  group’s  members. 

4-1.2  Familiarity  with  Computers/Affinity  Towards  Computers.  Before  the  study  began, 
the  researcher  wanted  to  determine  the  computer  experience  levels  of  the  participants,  as  well  as 
their  “comfort  levels”  in  using  a  personal  computer  (PC).  Figure  4.3  provides  a  good  overview 
of  the  responses  concerning  experience  levels  with  PCs.  Most  of  the  participants  (91%)  used  a 
computer  at  work,  while  83%  had  computer  training,  and  60%  actually  used  a  PC  at  home.  These 
figures  reflect  a  high  degree  of  familiarity  with  computer  technology. 

Knowledge  of  the  comfort  levels  was  needed  to  determine  whether  or  not  any  participants 
were  computer  averse.  The  participants’  “comfort”  level  with  computer  technology  was  actually 
fairly  high.  The  overall  sample  median  was  “moderately  comfortable”  (the  second  highest  response 
level),  and  the  most  frequent  response  was  “extremely  comfortable”  (reference  Table  4.3).  A  more 
important  aspect  in  this  area,  though,  was  the  evaluation  of  the  comfort  level  of  those  groups  which 
were  exposed  to  the  GDSS. 

2Thirty-three  people  (94%)  responded  to  this  question. 
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Figure  4.3.  Comparison  of  Computer  Familiarity  Across  Groups. 

Groups  1,  2,  and  3  used  a  GDSS  as  part  of  Experiment  1.  Substantial  affinity  or  aversion  to 
computers  in  these  three  groups  could  have  potentially  confounded  the  experiment’s  results.  None 
of  these  three  groups  were  considered  computer-averse  as  indicated  by  the  median  and  modal  values 
provided  in  Table  4.3.  Group  1  had  the  highest  comfort  levels,  and  Groups  2  and  3  were  roughly 
equivalent. 


Table  4.3.  Comparison  of  Comfort  Using  a  PC  Across  Groups. 


COMFORT  LEVEL 

INDEX 

GP  I 

GP  2 

GP  3 

GP  4 

GP  8 

GP  6 

GP  7 

Extremely  Uncomfortable 

1 

0 

0 

0 

0 

1 

0 

0 

Moderately  Uncomfortable 

2 

0 

0 

1 

0 

0 

0 

0 

Barely  Uncomfortable 

3 

0 

0 

0 

0 

0 

0 

0 

Borderline 

4 

0 

2 

1 

0 

0 

0 

0 

Barely  Comfortable 

5 

1 

0 

0 

1 

1 

0 

1 

Moderately  Comfortable 

6 

1 

2 

2 

1 

3 

2 

1 

Extremely  Comfortable 

7 

3 

1 

1 

3 

0 

3 

3 

SAMPLE  MEDIAN 

7 

6 

6 

7 

6 

7 

7 

SAMPLE  MODE 

7 

4/6 

6 

7 

6 

7 

7 

4-2  Complications  in  the  Experimental  Process 

There  were  several  complications  which  arose  during  the  actual  experimental  process.  They 
are  described  below. 
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•  As  part  of  Experiment  1  on  both  days,  one  of  the  two  GDSS  facilitators  abandoned  the 
structured  process  after  completing  only  one  career  field  (i.e.,  50  of  100  decisions  each  day). 
This  action  was  taken  for  real  world  expediency  reasons — to  complete  all  the  decision  tasks 
within  a  reasonable  amount  of  time.3  This  action  meant  that  observations  from  intervals 
6-10  for  Group  1  and  intervals  16-20  for  Group  2  were  not  recorded  and,  therefore,  could 
not  be  used  in  the  decision  speed  analysis.  The  sample  size  for  the  consensus  analysis  in 
Experiment  1  was  reduced,  accordingly.  There  was  no  way  to  determine  whether  this  action 
confounded  the  subjective  measures  obtained  from  participant’s  responses  on  the  post-process 
questionnaire. 

•  On  Day  2,  Group  4  could  not  distinguish  between  the  Procurement  and  Contracting  career 
fields.  They  considered  them  to  be  one-in-the-same;  consequently,  this  group  did  not  vote  on 
their  last  career  field.  As  before,  this  resulted  in  loss  of  comparison  data  (i.e.,  50  decisions) 
for  intervals  16-20  for  the  decision  speed  and  consensus  analyses  for  both  Experiments  1  and 
2.  Again,  there  was  no  way  of  knowing  if  this  action  confounded  the  Group  4  responses  on 
the  post-process  questionnaire. 

•  The  facilitator  for  Group  2  did  not  record  interval  times  for  the  first  five  intervals  on  Day  1 
(i.e.,  Career  Field  1),  but  did  so  for  the  remaining  15  intervals.  This  event  also  resulted  in 
the  loss  of  comparison  data  (i.e.,  50  decisions)  for  decision  speed  only.  Consensus  data  was 
unaffected. 

•  An  inadvertent  assignment  of  different  facilitators  (i.e.,  SI  and  EM)  to  Group  4,  the  control 
group  for  Experiment  1,  between  the  two  days  could  have  confounded  the  decision  speed 
interval  data.  The  original  intent  was  to  maintain  the  same  facilitator  for  this  control  group 
for  both  days  to  preclude  introducing  different  facilitator  effects.  Plotting  the  equations  for  SI 
and  EM  observations  (reference  Figure  4.4)  over  the  entire  first  day’s  experiment  (intervals 
1-10)  reveals  two  practically  identical  curves,  as  shown  in  Figure  4.5,  with  a  correlation 
coefficient  of  r  =  .96371.  From  the  data  presented,  SI  and  EM  1-10  interval  times  were 
practically  identical.  Consequently,  SI-EM  facilitator  effects  for  intervals  1-10,  and  therefore 
this  discrepancy,  were  deemed  negligible.  The  SAS  correlation  data  along  with  the  equations 
for  the  curves,  calculated  using  Malhemaiica,  are  provided  in  Appendix  D. 

3The  primary  objectives  of  the  conference  took  precedence  over  the  research  objectives. 
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Figure  4.4.  SI-EM  Observations  for  Intervals  1-10. 


Figure  4.5.  SI-EM  Fitted  Curves  for  Intervals  1-10. 
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f.S  Results  of  the  Statistical  Analysis  on  Performance  Outcomes 

The  first  performance  outcome  variable  evaluated  was  Decision  Speed.  Decision  speed  was 
tested  in  terms  of  the  time  observed  to  complete  each  of  the  20  decision  intervals.  The  final 
regression  model  for  the  two  experiments  met  the  assumptions  outlined  earlier  in  Chapter  III:  ho- 
moscedasticity,  no  error  autocorrelation,  and  residual  normality.  The  final  model  was  also  evaluated 
for  multicollinearity  and  the  presence  of  outliers.  The  recorded  interval  times  for  both  experiments 
are  provided  in  Appendix  E,  and  a  compilation  of  the  applicable  SAS  results  are  provided  in 
Appendix  F. 

In  order  to  test  the  regression  assumptions  and  arrive  at  a  final  model  for  decision  speed, 
several  interim  models  were  first  considered.  Four  models  were  evaluated  for  Experiment  1,  and 
two  were  evaluated  for  Experiment  2.  The  models  evaluated  for  Experiment  1  are  defined  as  follows: 

•  Model  1A:  Decision  Speed  =  Intercept  +  Group  +  Method  +  Interval  +  Error 

•  Model  IB:  Decision  Speed  =  Intercept  +  Group  +  Method  +  Interval  +  Facilitator  +  Error 

•  Model  1C:  Decision  Speed  Transformed  =  Intercept  +  Group  +  Method  +  Interval  +  Error 

•  Model  ID:  Decision  Speed  Transformed  =  Intercept  +  Group  +  Method  +  Interval  +  Facilitator  +  Error 

The  models  evaluated  for  Experiment  2  were  defined  as  follows: 

•  Model  2A:  Decision  Speed  =  Intercept  +  Group  +  Method  +  Interval  +  Error 

•  Model  2B;  Decision  Speed  Transformed  =  Intercept  +  Group  +  Method  +  Interval  +  Error 

Models  1C  and  2B,  both  using  transformed  observation  data,  were  ultimately  selected  as  the 
final  regression  models.  The  following  discussion  addresses  the  assumptions  which  were  tested, 
leading  up  to  the  selection  of  the  final  regression  models. 

Regression  analysis  assumes  a  finite  variance  for  the  distribution  of  residuals,  the  error  terms, 
which  should  be  constant  for  all  values  of  the  independent  variables.  This  “constancy”  of  the 
residual  variance  is  called  homoscedasticity  (27:183).  Examining  the  distribution  of  the  residuals 
for  both  experiments  (reference  the  residual-predicted  value  plots  in  Appendix  F)  revealed  that 
the  residual  variance  increased  as  the  decision  time  increased.  This  condition  was  interpreted  as 
residual  heteroscedasticity.  Consequently,  the  observations  were  transformed  using  a  square  root 
transformation  which  did,  in  fact,  stabilize  the  residual  variance  (27:132-138).  Specifically,  this 
procedure  reduced  the  variance  associated  with  the  larger  interval  times  more  than  the  variance 
associated  with  the  smaller  interval  times,  ultimately  correcting  the  heteroscedasticity  observed  in 
the  original  models  (i.e.,  models  1A  and  2A).  For  example,  the  plot  of  residuals  against  predicted 
values  showed  a  random  scattering  of  the  residuals  in  a  horizontal  band  about  the  prediction  line, 
meaning  constancy  of  the  residuals.  Additionally,  this  plot  confirmed  that  neither  of  the  models 
required  quadratic  terms. 
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The  Durbin- Watson  D  test  for  autocorrelation  was  performed,  and  the  D  statistic  exceeded 
the  upper  bound  da  for  every  model  before  and  after  the  data  transformation  (reference  Table  4.4). 
Following  transformation  of  the  data,  the  D-W  d  statistic  improved,  exceeding  2.0 — a  clear  indica¬ 
tion  of  no  autocorrelation. 


Table  4.4.  Results  of  Regression  Analysis:  Assumptions  Concerning  Residuals. 


MODEL 

DURBIN. 

WATSON  O/du 

RESIDUAL  AUTO¬ 
CORRELATION? 

SHAPIRO- 

WILK 

RE8IDUAL 

NORMALITY? 

RESIDUAL  HOMO- 

SCEDASTICITY? 

1A 

1.933/1.81 

No 

.9105 

Yea 

Questionable 

IB 

1.988/1.85 

No 

.9014 

Yea 

Questionable 

1C 

3-027/1. 81 

No 

.9667 

Yea 

Yea 

ID 

3-134/1 .85 

No 

.9603 

Yea 

Yea 

2A 

1  961/1.80 

No 

.9074 

Yea 

2B 

3. 219/1. 80 

No 

.9736 

Yea 

Yea 

The  residuals  for  both  experiments  were  normally  distributed — the  Shapiro-Wilk  statistic 
exceeded  .9  for  every  model  (reference  Table  4.4).  Since  the  presence  of  outliers  increased  the 
skewness  and  kurtosis  of  the  residual  distribution,  transformation  of  the  data  reduced  these  effects. 

Additional  diagnostic  procedures,  such  as  studentized  residuals,  normality  probability  plots, 
and  residual-predicted  value  plots,  revealed  the  presence  of  outliers  in  the  data.  Two  or  three  high 
value  outliers  (i.e.,  studentized  residuals  greater  than  +2.5)  were  detected.  These  extreme  vadues 
become  readily  apparent  when  viewed  on  a  time  series  line  plot  of  the  various  groups.4  Referencing 
Figure  4.6  for  Experiment  1  and  Figure  4.7  for  Experiment  2,  the  data  points  corresponding  to 
these  outliers  are  easily  seen:  Group  1  at  Interval  2,  Group  2  at  Interval  11  (i.e.,  the  first  interval  of 
Day  2),  and  Groups  4,  5,  6  at  Interval  1.  These  spikes  in  the  decision  time  series  plots  are  assumed 
to  be  indications  of  learning  effects,  as  the  group  members  were  being  introduced  to  a  new  group 
decision  making  process.  Consequently,  they  were  retained  in  the  model.  Although  there  were 
several  low  value  outliers  (i.e.,  studentized  residuals  less  than  -2.5),  there  presence  did  not  appear 
to  be  erroneous,  and  they  were  also  retained  in  the  model. 

An  independent  variable  for  the  facilitator  was  added  to  determine  its  effect  on  the  model 
for  Experiment  1  (i.e.,  models  IB  and  ID).  The  p-value  for  this  variable  was  not  statistically 
significant  (p  =  .4370/.2272,  a  =  .05),  and  its  inclusion  in  the  model  resulted  in  a  significant  level 
of  multicollinearity  (reference  Table  4.5  and  Appendix  F).  The  presence  of  the  facilitator  term 
appeared  to  be  measuring  the  same  phenomena  as  that  of  the  group  variables.  Also,  the  presence 
of  the  facilitator  variable  had  little  effect  on  the  p-value  for  the  GDSS  method  being  tested.  For 

4  Discontinuities  in  the  line  plots  are  due  to  data  being  lost  during  the  experimentation,  as  addressed  in  Compli¬ 
cations  in  the  Experimental  Process.  An  anomylous  feature  of  these  plots  is  the  inclusion  of  a  line  drawn  to  or  from 
the  x-axis  whenever  data  is  lost  or  restored,  respectively.  The  reader  should  not  interpret  the  data  as  zero  values, 
but  should  ignore  these  lines. 
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Figure  4.7.  Time  Series  Plot  of  Decision  Time  for  Experiment  2. 
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these  reasons,  the  facilitator  variable  was  dropped  from  consideration  in  the  regression  model.  No 
other  independent  variable  in  the  remaining  models  had  an  appreciable  amount  of  multicollinearity. 


Table  4.5.  Results  of  Regression  Analysis:  Multicollinearity. 


MODEL 

MODEL  VIF 

HIGHEST  INDEP 

VARIABLE  VIF 

SIGNIFICANT 

MULTICOLLINEARITY? 

1A 

2.5265 

4.7168 

No 

IB 

2.5562 

32.6882 

Yes 

1C 

3.0230 

4.7168 

No 

ID 

3.1104 

32.6882 

Yes 

2A 

1.7655 

2.6346 

No 

2B 

1.9535 

2.6346 

No 

4-3.1  Decision  Speed. 

4.3.1. 1  Experiment  1,  GDSS  Effects.  The  research  objective  was  to  compare  GDSS 
and  non-GDSS  (facilitated  manual)  groups  with  respect  to  decision  speed  to  evaluate  GDSS  effects. 
Decision  speed  was  measured  in  terms  of  the  time  to  complete  intervals  consisting  of  10  decisions 
each.  Regression  analysis  using  indicator  variables  was  used.  The  hypotheses  were: 

Ho'-  There  is  not  a  Statistically  Significant  Relationship  between  the  Decision  Method  (GDSS)  and  Decision 

Speed. 


Ha\  There  is  a  Statistically  Significant  Relationship  between  the  Decision  Method  (GDSS)  and  Decision 

Speed. 

The  null  hypothesis  was  rejected  (p  =  .0096,  a  =  .05),  and  it  was  concluded  that  there 
was  a  significant  relationship  between  the  decision  making  method,  facilitated  GDSS,  and  decision 
speed.  It  took  facilitated  GDSS  groups  longer  to  complete  the  decision  making  tasks  than  facilitated 
manual  groups.  The  regression  analysis  results  for  GDSS  effects  on  decision  speed  and  all  regression 
coefficients  are  shown  in  Table  4.6  and  Appendix  F.5  6 


4.3. 1.2  Experiment  2,  Structure  Effects.  The  research  objective  was  to  compare  fa¬ 
cilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect  to  decision  speed  to 


^Concerning  the  interpretation  of  the  parameter  estimates,  recall  that  this  regression  analysis  used  indicator 
variables  for  the  groups  in  comparison  with  a  control  group  (i.e.,  Group  4).  For  this  reason,  no  parameter  estimates 
are  provided  for  the  control  group. 

®Although  regression  analysis  was  not  used  here  to  provide  a  validated  prediction  model,  the  model  does  provide 
a  good  fit  (reference  Appendix  F).  Predicted  values  for  decision  time  can  be  obtained  by  substituting  the  values  for 
the  independent  variables  into  the  prediction  regression  model,  Y  ,  and  squaring  the  result.  The  answer  will  be  in 
the  original  units  of  minutes. 
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Table  4.6.  Results:  Experiment  1,  Decision  Speed — Regression  Analysis. 


VARIABLE 

PARAMETER 

ESTIMATE 

MSOS 

p  Value 

DECISION 

Intercept 

5.0041 

20.470 

.0001 

Reject  H0 

Group  1 

-0.0707 

0.264 

.7930 

Fail  to  Reject  H0 

Group  2 

-0.2496 

0.790 

.4330 

Fail  to  Reject  H0 

Group  3 

-1.3064 

3.371 

.0014 

Reject  H0 

Day 

-0.5663 

3.187 

.0024 

Reject  H0 

Interval 

-0.2265 

6.800 

.0001 

Reject  H0 

Method 

0.8007 

2.687 

.0096 

Reject  Ho 

evaluate  structure  effects.  Decision  speed  was  measured  in  terms  of  the  time  to  complete  inter¬ 
vals  consisting  of  10  decisions  each.  Regression  analysis  using  indicator  variables  was  used.  The 
hypotheses  were: 

Ho :  There  is  not  a  Statistically  Significant  Relationship  between  the  Decision  Method  (FAC-MAN)  and 
Decision  Speed. 

Ha :  There  is  a  Statistically  Significant  Relationship  between  the  Decision  Method  (FAC-MAN)  and  Decision 

Speed. 

The  p- value  for  the  FAC-MAN  method  actually  improved  (from  .0658  to  .0265)  as  a  result 
of  the  transformation  process.  The  null  hypothesis  was,  therefore,  rejected  ( p  =  .0265,  o  =  .05), 
and  it  was  concluded  that  there  was  a  significant  relationship  between  the  decision  making  method, 
facilitated  manual,  and  decision  speed.  It  took  facilitated  manual  groups  longer  to  complete  the  de¬ 
cision  making  tasks  than  unfacilitated  manual  groups.  The  regression  analysis  results  for  FAC-MAN 
effects  on  decision  speed  and  all  regression  coefficients  are  shown  in  Table  4.7  and  Appendix  F.7 


4-3.2  Consensus. 


4 .3-2.1  Experiment  1,  GDSS  Effects.  The  research  objective  was  to  compare  GDSS 
and  non-GDSS  groups  with  respect  to  consensus  to  evaluate  GDSS  effects.  Two  statistical  proce¬ 
dures  were  performed  examining:  1)  the  level  of  consensus  in  terms  of  the  number  of  the  total  tasks 
for  which  the  group  actually  reached  consensus,  and  2)  the  ability  to  reach  consensus  in  terms  of 
the  number  of  votes  it  took  to  obtain  consensus  for  each  task.  The  distributions  between  GDSS 


Concerning  the  interpretation  of  the  parameter  estimates,  recall  that  this  regression  analysis  used  indicator 
variables  for  the  groups  in  comparison  with  a  control  group  (i.e.,  Group  7).  For  this  reason,  no  parameter  estimates 
are  provided  for  the  control  group. 
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Table  4.7.  Results:  Experiment  2,  Decision  Speed — Regression  Analysis. 


VARIABLE 

PARAMETER 

ESTIMATE 

p  Value 

DECISION 

Intercept 

3.7554 

13.826 

.0001 

Reject  H0 

Group  4 

0.4178 

1.127 

.2636 

Fail  to  Reject  Ha 

Group  5 

0.2833 

1.013 

.3147 

Fail  to  Reject  H0 

Group  6 

0.4715 

1.686 

.0964 

Fail  to  Reject  Ha 

Day 

-0.1159 

0.625 

.5339 

Fail  to  Reject  H0 

Interval 

-0.2022 

6.286 

.0001 

Reject  H0 

Method 

0.5676 

2.269 

.0265 

Reject  H0 

and  FAC-MAN  were  compared  to  determine  whether  or  not  a  significant  difference  existed.  A 
contingency  table  was  developed  with  two  classes  of  consensus:  “Yes”  consensus  was  reached,  and 
“No”  consensus  was  not  reached.  Chi-Square  and  Fisher’s  Exact  tests  were  performed,  and  the 
hypotheses  were: 

H0-  There  is  No  Difference  between  GDSS  and  FAC-MAN  in  Relationship  to  the  Level  of  Consensus. 

H a :  There  is  a  Difference  between  GDSS  and  FAC-MAN  in  Relationship  to  the  Level  of  Consensus. 

The  null  hypothesis  concerning  level  of  consensus  was  rejected  (p  =  .042,  a  =  .05),  and  it 
was  concluded  that  the  level  of  consensus  between  facilitated  GDSS  groups  and  facilitated  manual 
groups  was  not  the  same.  The  level  of  consensus  in  facilitated  GDSS  groups  was  proportionally  lower 
than  facilitated  manual  groups.  The  contingency  table  for  Experiment  1  was  based  on  facilitated 
GDSS  groups  reaching  consensus  for  276  of  300  (92%)  tasks;  whereas,  facilitated  manual  groups 
reached  consensus  for  526  of  550  (96%)  tasks.  Table  4.8  provides  the  statistical  results.  Recorded 
consensus  data  is  provided  in  Appendix  G.  Contingency  tables/Chi-Square  test  results  are  provided 
in  Appendix  H,  and  Fisher’s  Exact  test  results  are  provided  in  Appendix  I. 


Table  4.8.  Results:  Experiment  1,  Level  of  Consensus — Chi-Square  Analysis. 


T  VALUE 

m 

X2  p  VALUE 

FISHER’S  EXACT 

p VALUE 

DECISION 

4.818 

3.841 

0.028 

0.042 

Reject  Ho 

The  second  statistical  procedure  concerning  Ability  to  Reach  Consensus  compared  the  dis¬ 
tributions  for  the  number  of  votes  it  took  groups  to  reach  consensus.  A  contingency  table  was 
developed  with  four  classes  of  the  number  of  votes  required  to  reach  consensus  for  each  task:  “1” 
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vote,  “2”  votes,  “3”  votes,  or  they  “Never”  reached  consensus.  Chi-Square  and  Fisher’s  Exact  test 
were  performed,  and  the  hypotheses  were: 

Ha:  There  is  No  Difference  between  GDSS  and  FAC-MAN  in  Relationship  to  the  Ability  to  Reach  Consensus. 

Ha-  There  is  a  Difference  between  GDSS  and  FAC-MAN  in  Relationship  to  the  Ability  to  Reach  Consensus. 

The  null  hypothesis  for  the  ability  of  a  group  to  reach  consensus  was  rejected  (p  =  .0006, 
a  =  .05),  and  it  was  concluded  that  the  ability  to  reach  consensus  between  facilitated  GDSS  and 
facilitated  manual  groups  was  not  the  same.  Proportionally,  facilitated  manual  groups  reached 
consensus  in  fewer  votes  than  did  facilitated  GDSS  groups.  Since  the  initial  vote  for  all  groups  was 
merely  a  starting  point,  no  statistical  significance  was  placed  on  this  vote.  However,  the  second 
vote  does  indicate  the  ability  to  reach  consensus.  The  data  from  the  contingency  tables  showed 
that  facilitated  manual  groups  converged  to  consensus  on  the  second  vote  more  frequently  (287  of 
550  tasks  or  52%)  than  did  facilitated  GDSS  groups  (120  of  300  tasks  or  40%).  The  proportion 
of  tasks  where  groups  reached  consensus  on  the  third  vote  were  similar  between  the  experimental 
conditions.  The  proportion  of  tasks  where  GDSS  groups  failed  to  reach  consensus  was  twice  that 
of  facilitated  manual  groups  (8%  versus  4%).  Table  4.9  provides  the  statistical  results.  Recorded 
consensus  data  is  provided  in  Appendix  G.  Contingency  tables/ Chi-Square  test  results  are  provided 
in  Appendix  H,  and  Fisher’s  Exact  test  results  are  provided  in  Appendix  I. 

Table  4.9.  Results:  Experiment  1,  Number  of  Votes  to  Consensus — Chi-Square  Analysis. 


T  VALUE 

l 

X2  p  VALUE 

FISHER’S  EXACT 

p  VALUE 

DECISION 

17.268 

1  7815 

.001 

.0006 

Reject  Ho 

4-3. 2. 2  Experiment  2,  Structure  Effects.  The  research  objective  was  to  compare  facili¬ 
tated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  with  respect  to  consensus  to  determine 
if  structure  effects  influenced  the  extent  to  which  groups  reached  consensus.  The  statistical  pro¬ 
cedure  examined  the  level  of  consensus  in  terms  of  the  total  tasks  for  which  the  group  actually 
reached  consensus.  The  distributions  between  GDSS  and  FAC-MAN  were  compared  to  determine 
whether  or  not  a  significant  difference  existed.  A  contingency  table  was  developed  with  two  classes 
of  consensus:  “Yes”  consensus  was  reached,  and  “No”  consensus  was  not  reached.  Chi-Square  and 
Fisher’s  Exact  tests  were  performed,  and  the  hypotheses  were: 

H0'-  There  is  No  Difference  between  FAC-MAN  and  Manual  in  Relationship  to  the  Level  of  Consensus. 

Ha'.  There  is  a  Difference  between  FAC-MAN  and  Manual  in  Relationship  to  the  Level  of  Consensus. 
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The  null  hypothesis  for  level  of  consensus  was  rejected  (p  =  .0006,  a  =  .05),  and  it  was 
concluded  that  the  level  of  consensus  between  facilitated  manual  and  unfacilitated  manual  groups 
was  not  the  same.  The  level  of  consensus  in  facilitated  manual  groups  was  proportionally  lower 
than  unfacilitated  manual  groups.  The  contingency  table  for  Experiment  2  was  based  on  facilitated 
manual  groups  reaching  consensus  for  526  of  550  (96%)  tasks;  whereas,  unfacilitated  manual  groups 
reached  consensus  for  397  of  400  (99%)  tasks.  Table  4.10  provides  the  statistical  results.  Recorded 
consensus  data  is  provided  in  Appendix  G.  Contingency  Tables/Chi-Square  test  results  are  provided 
in  Appendix  H,  and  Fisher’s  Exact  test  results  are  provided  in  Appendix  I. 


Table  4.10.  Results:  Experiment  2,  Level  of  Consensus — Chi-Square  Analysis. 


T  VALUE 

X2  p  VALUE 

FISHER’S  EXACT 

p  VALUE 

DECISION 

10.951 

3.841 

0.001 

0.0006 

Reject  H0 

4-3.3  Discussion. 

4-3.3. 1  Decision  Speed.  Referencing  Figures  4.8  and  4.9,  it  was  interesting  to  note  that 
whenever  a  group  transitioned  from  unfacilitated  manual  to  facilitated  manual  or  from  facilitated 
manual  to  facilitated  GDSS,  decision  time  increased  dramatically,  without  exception.  TVansitions 
in  the  opposite  direction  did  not  experience  such  an  increase.  For  example,  in  Figure  4.9  Group 
6  transitioned  from  Manual  to  FAC-MAN  with  a  dramatic  increase  in  decision  time  (see  points 
“c”  to  “d”),  but  Group  5’s  decision  time  did  not  respond  to  the  same  degree  when  moving  from 
FAC-MAN  to  Manual  (see  points  “a”  to  “b”).  Likewise,  in  Figure  4.8  Group  2  experienced  a 
significant  increase  in  decision  time  when  transitioning  from  FAC-MAN  to  GDSS  (see  points  “c” 
to  “d”).  Group  1,  however,  did  not  experience  any  increase  moving  from  GDSS  to  FAC-MAN  (see 
points  “a”  to  “b”).  Each  time  a  group  experienced  either  the  GDSS  or  FAC-MAN  experimental 
conditions,  initial  training  was  provided  to  explain  the  procedures  and  to  conduct  a  trial  run. 
The  time  used  for  this  training  period  was  not  included  in  any  of  the  interval  data.  One  might 
have  expected  a  slight  increase  in  time  for  GDSS  groups  to  become  familiar  with  the  operation  of 
a  computer;  however,  it  was  less  clear  why  decision  time  increased  for  groups  transitioning  to  a 
facilitated  manual  process  which  used  voting  cards. 

The  results  from  the  regression  analysis  showed  that  facilitated  GDSS  groups  were  slower  than 
facilitated  manual  groups  which,  in  turn,  were  slower  than  unfacilitated  manual  groups.  There 
is  a  clear  indication  that  process  structure  did  adversely  affect  decision  speed;  however,  group 
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Figure  4.8.  Transitions  in  Experimental  Condition  for  Experiment  1. 
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Figure  4.9.  Transitions  in  Experimental  Condition  for  Experiment  2. 
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dynamics  probably  also  contributed  to  the  differences  experienced  in  the  facilitated  manual  and 
facilitated  GDSS  groups.  For  example,  note  in  Figure  4.6  Group  3,  a  facilitated  GDSS  group  for 
both  days,  experienced  the  fastest  decision  speed  of  all  groups  in  Experiment  1.  Additionally, 
Group  3  experienced  half  the  learning  effect  of  the  other  GDSS  groups,  and  was  faster  than  Group 
4,  the  facilitated  manual  control  group,  for  all  but  three  intervals.  Probably  due  to  structure 
effects,  Group  3  was  slower  than  Group  7,  the  unfacilitated  manual  group.  It  is  not  unknown 
whether  Group  7  was  faster  than  all  other  groups  because  of  the  demographic  factors:  they  knew 
more  group  members  than  any  other  group.  Group  7  had  fewer  years  of  experience  in  their  career 
field,  however.  These  factors  may  have  led  to  fewer  disagreements  or  even  less  thorough  discussions 
which  could  have  resulted  in  improved  decision  speed;  however,  the  converse  could  also  be  true. 

There  were  two  interesting  observations  concerning  other  independent  variables.  The  vari¬ 
able  “Interval”  was  statistically  significant  for  both  experiments  (p=.0001,  a  =  .05).  Its  negative 
parameter  estimates  indicated  that  as  groups  moved  to  later  intervals,  the  time  to  complete  these 
intervals  decreased.  Therefore,  decision  speed  improved  as  the  experiments  progressed.  Also,  the 
variable  “Day”  with  its  negative  parameter  estimate  was  statistically  significant  for  Experiment  1 
(p  =  .0024.  a  =  .05),  indicating  that  groups  were  faster  on  the  second  day.  Both  of  these  find¬ 
ings  support  the  conclusion  that  some  amount  of  learning  effect  was  present  in  the  group  decision 
making  processes. 

4-3. 3. 2  Consensus.  The  results  from  the  contingency  table/Chi-Square  tests  showed 
that,  proportionally,  facilitated  GDSS  groups  reached  consensus  less  than  facilitated  manual  groups, 
which  experienced  proportionally  lower  consensus  than  unfacilitated  manual  groups.  It  also  took 
facilitated  GDSS  groups  more  votes  proportionally  to  reach  consensus  than  it  did  for  facilitated 
manual  grovps  These  observations  suggest  that  structure  does  adversely  affect  consensus;  however, 
group  dynamics  probably  contributed  to  the  significant  differences  in  the  outcomes  of  the  facilitated 
GDSS  and  facilitated  manual  groups. 


4-4  Results  of  the  Statistical  Analysis  on  Secondary  Outcomes 


4-4- 1  User  Responses  Concerning  Satisfaction  and  Perceptions. 

4. 4.1.1  Experiment  1,  GDSS  Effects.  The  research  objective  was  to  compare  GDSS 
and  non-GDSS  groups  to  evaluate  GDSS  effects  with  respect  to  eight  dependent  variables  of  interest. 
These  variables  were  stated  in  terms  of: 
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•  The  user’s  satisfaction  with  the  decision  making  process. 

•  The  user’s  satisfaction  with  the  results  of  the  decision  making  process. 

•  The  user’s  perception  of  task  difficulty. 

•  The  user’s  perception  of  the  group’s  ability  to  stay  task-focused. 

•  The  user’s  perception  of  productivity  in  terms  of  time  efficiency. 

•  The  user's  perception  of  productivity  in  terms  of  ability  to  reach  consensus. 

•  The  user’s  perception  of  the  group’s  level  of  conflict. 

•  The  user’s  perception  of  the  level  of  process  structure. 

To  measure  the  statistical  significance  of  the  experimental  conditions  with  regard  to  the  eight 
dependent  variables  of  interest,  two  sets  of  tests  were  performed:  Contingency  table/Chi-Square 
and  Sign  tests. 

The  contingency  table/Chi-Square  tests  were  used  to  determine  whether  or  not  a  significant 
difference  existed  between  the  distribution  of  responses  for  GDSS  and  FAC-MAN.  These  distribu¬ 
tions  were  based  on  questionnaire  responses  across  all  GDSS  and  facilitated  manual  sessions  for 
each  day  for  every  dependent  variable  of  interest.  Responses  for  each  dependent  variable  of  interest 
were  classified  into  seven  categories,  from  “1”  meaning  an  extremely  unfavorable  response  to  “7” 
meaning  an  extremely  favorable  response.  Chi-Square  and  Fisher’s  Exact  tests  were  performed, 
and  the  hypotheses  were: 

Ho-  There  is  No  Difference  between  GDSS  and  FAC-MAN  in  Relationship  to  the  Dependent  Variable  of 
Interest. 

Ha:  There  is  a  Difference  between  GDSS  and  FAC-MAN  in  Relationship  to  the  Dependent  Variable  of  Interest. 

Based  on  the  Chi-Square  and  Fisher’s  Exact  tests,  the  null  hypothesis  could  not  be  rejected 
for  any  of  the  dependent  variables  of  interest  for  either  day  (reference  Tables  411  and  4.12).  From 
these  results,  it  was  concluded  that  there  was  no  difference  between  facilitated  GDSS  and  facilitated 
manual  with  respect  to: 

•  The  user’s  satisfaction  with  the  decision  making  process  (p  =  .288  and  .614,  a  =  .05). 

•  The  user’s  satisfaction  with  the  results  of  the  decision  making  process  (p  =  1.000  and  .888, 

a  =  .05). 

•  The  user’s  perception  of  task  difficulty  (p  =  .972  and  .907,  a  =  .05). 

•  The  user’s  perception  of  the  group’s  ability  to  stay  task-focused  (p  =  .811  and  .799,  a  =  .05). 

•  The  user's  perception  of  productivity  in  terms  of  time  efficiency  (p  =  .097  and  .890,  a  =  .05). 

•  The  user’s  perception  of  productivity  in  terms  of  ability  to  reach  consensus  (p  =  .317  and 

.460,  q  =  .05). 

•  The  user's  perception  of  the  group’s  level  of  conflict  (p  =  .932  and  .791,  a  =  .05). 

•  The  user’s  perception  of  the  level  of  process  structure  (p  =  1.000  and  .835,  o  =  .05). 


4-18 


User  responses  recorded  on  the  post-process  questionnaires  are  summarized  in  tabular  form 
in  Appendix  J.  The  contingency  tabies/Chi-Square  test  results  are  provided  in  Appendix  K,  and 
the  Fisher’s  Exact  test  results  are  shown  in  Appendix  L. 


Table  4.11.  Results:  Experiment  1,  Day  1  Participant  Responses- Chi-Square  Analysis. 


VARIABLE 

T  VALUE 

Hi 

x 2  p  VALUE 

FISHER’S  EXACT 

p VALUE 

DECISION 

Satisfied  with  Process 

2  670 

7.815 

0.2C2 

0.288 

Fail  to  n eject  Ho 

Satisfied  with  Results 

0  110 

3  841 

0.734 

1.000 

Fail  to  Reject  H0 

Task  Difficulty 

2  457 

11.070 

0.783 

0.972 

Fail  to  Reject  Ha 

Stay  Task-Focused 

1.711 

7.815 

0.634 

0.811 

Flail  to  Reject  Ho 

Time  Efficiency 

8.923 

12.592 

0.178 

0.097 

Ffeil  to  Reject  l/0 

Reach  Consensus 

3.299 

7815 

0.348 

0.317 

Fail  to  Reject  Ho 

Level  of  Conflict 

2  649 

11.070 

0.754 

0.932 

Fail  to  Reject  Ha 

Level  of  Structure 

0.446 

5.991 

0.800 

1.000 

Fail  to  Reject  Ho 

Table  4.12.  Results:  Experiment  1,  Day  2  Participant  Responses— Chi-Square  Analysis. 


VARIABLE 

T  VALUE 

— 

X2  p  VALUE 

FISHER’S  EXACT 

p  VALUE 

DECISION 

Satisfied  with  Process 

3  889 

9.488 

0.421 

0.614 

Fail  to  Reject  H0 

Satisfied  with  Results 

1  891 

7.815 

0.596 

0.888 

Fail  to  Reject  Ho 

Task  Difficulty 

mmm 

11.070 

0  813 

0  907 

Fail  to  Reject  //„ 

Slay  Task-Focused 

0.926 

5.991 

0.629 

0  799 

Fail  to  Reject  H0 

Tune  Efficiency 

1.007 

7.615 

0.644 

0  890 

Fbi*  to  Reject  H0 

Reach  Consensus 

1.930 

5.991 

0.381 

0.460 

Fail  to  Reject  Ho 

Level  of  Conflict 

3.914 

11070 

0.562 

0  791 

Fail  to  Reject  //„ 

I^cvel  of  Structure 

0.700 

5.991 

0.703 

0.835 

Fail  to  Reject  Ho 

The  Sign  test  was  used  to  determine  if  the  GDSS  treatment  yielded  higher  responses  for  each 
dependent  variable  of  interest  than  FAC-MAN  for  those  groups  which  experienced  both  experi¬ 
mental  conditions  over  the  t.wo  days  (i.c. ,  cross-over  Groups  1  and  2).  The  Sign  test  compared 
the  response  levels  (i.e.,  17)  by  participant  for  both  an  objective  test  and  a  subjective  test.  The 
objective  test  compared  an  individual’s  responses  for  an  experimental  condition  experienced  and 
reported  on  day  1  to  the  individual’s  responses  for  an  alternate  experimental  condition  experienced 
and  reported  on  day  2.  The  subjective  test,  however,  was  based  on  the  individual’s  direct  compari¬ 
son  between  the  two  experimental  conditions.  The  hypotheses  for  both  the  objective  and  subjective 
tests  were: 

//„  '  p{  +  )  <  p(  — )  or  GDSS  <  FAC-MAN  for  the  Dependent  Variable  of  Interest. 

//«  ;  p(  +  )  >  ;>{“)  or  GDSS  >  FAC-MAN  for  the  Dependent  Variable  of  Interest. 


4-19 


Based  on  the  Sign  tests  for  the  objective  tests,  the  null  hypothesis  could  not  be  rejected 
for  any  dependent  variable  of  interest.  It  was  concluded  that  facilitated  GDSS  groups  did  not 
experience  higher  levels  of  response  for  each  of  the  dependent  variables  of  interest  (reference  the  p 
values  in  Table  4.13,  a  =  .05).  Sign  test  calculations  are  provided  in  Appendix  M. 

Based  on  the  Sign  tests  for  the  subjective  tests,  the  null  hypothesis  was  rejected  for  task 
difficulty  (p  =  .0156,  a  =  .05)  as  shown  in  Table  4.13;  however,  this  result  conflicts  with  that  from 
the  objective  test.  It  was  concluded  that  the  facilitated  GDSS  process  was  perceived  as  being  more 
difficult,  but  the  results  are  conflicting. 


Table  4.13.  Results:  Experiment  1,  Participant  Responses — Sign  Test  Analysis. 


OBJECTIVE 

SUBJECTIVE 

VARIABLE 

p VALUE 

DECISION 

p  VALUE 

DECISION 

Satisfied  with  Process 

0.9680 

Fail  to  Reject  H0 

0.8750 

Fail  to  Reject  Ho 

Satisfied  with  Results 

0.6875 

Fail  to  Reject  H0 

0  8906 

Fail  to  Reject  Ho 

Task  Difficulty 

0.0898 

Fail  to  Reject  H0 

0.0156 

Reject  Ho 

Stay  Task-Focused 

0  9844 

Fail  to  Reject  H0 

0  1250 

Fail  to  Reject  H0 

Time  Efficiency 

0  9844 

Fail  to  Reject  H0 

0.9375 

Fail  to  Reject  H0 

Reach  Consensus 

0  6875 

Fail  to  Reject  H0 

0.6875 

Fail  to  Reject  Ho 

Level  of  Conflict 

0.3437 

Fail  to  Reject  Ho 

0.1250 

Fail  to  Reject  Ho 

Level  of  Structure 

0.2266 

Fail  to  Reject  Ho 

0  5000 

Fail  to  Reject  Ho 

4. 4. 1.2  Experiment  2,  Structure  Effects.  The  research  objective  was  to  compare  fa¬ 
cilitated  manual  (FAC-MAN)  and  unfacilitated  manual  groups  to  evaluate  structure  effects  with 
respect  to  eight  dependent  variables  of  interest.  These  variables  were  stated  in  terms  of: 

•  The  user’s  satisfaction  with  the  decision  making  process. 

•  The  user’s  satisfaction  with  the  results  of  the  decision  making  process. 

•  The  user’s  perception  of  task  difficulty. 

•  The  user’s  perception  of  the  group’s  ability  to  stay  task-focused. 

•  The  user’s  perception  of  productivity  in  terms  of  time  efficiency. 

•  The  user’s  perception  of  productivity  in  terms  of  ability  to  reach  consensus. 

•  The  user’s  perception  of  the  group’s  level  of  conflict. 

•  The  user’s  perception  of  the  level  of  process  structure. 

To  measure  statistical  significance  of  the  experimental  conditions  with  regard  to  the  eight 
dependent  variables  of  interest,  two  sets  of  tests  were  performed.  Again,  contingency  table/Chi- 
Square  tests  were  used  to  determine  whether  or  not  a  significant  difference  existed  between  the 
distribution  of  responses  for  FAC-MAN  and  Manual.  These  distributions  were  based  on  question¬ 
naire  responses  across  all  facilitated  manual  and  unfacilitated  manual  sessions  for  each  day  for 
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every  dependent  variable  of  interest.  Responses  for  each  dependent  variable  of  interest  were  clas¬ 
sified  into  seven  categories:  from  “1”  meaning  an  extremely  unfavorable  response  to  “7”  meaning 
an  extremely  favorable  response.  Chi-Square  and  Fisher’s  Exact  tests  were  performed,  and  the 
hypotheses  were: 

H0i  There  is  No  Difference  between  FAC-MAN  and  Manual  in  Relationship  to  the  Dependent  Variable  o >/ 
Interest. 

Ha :  There  is  a  Difference  between  FAC-MAN  and  Manual  in  Relationship  to  the  Dependent  Variable  of 
Interest. 

Based  on  the  Chi-Square  and  Fisher’s  Exact  tests,  the  null  hypothesis  could  not  be  rejected  for 
any  of  the  dependent  variables  of  interest  for  either  day  (reference  Tables  4.14  and  4.15).  From  these 
results,  it  was  concluded  that  there  was  no  difference  between  facilitated  manual  and  unfacilitated 
manual  groups  with  respect  to: 

•  The  user’s  satisfaction  with  the  decision  making  process  (p  =  .659  and  .654,  a  =  .05). 

•  The  user’s  satisfaction  with  the  results  of  the  decision  making  process  (p  =  .111  and  .884, 

o  =  .05). 

•  The  user’s  perception  of  task  difficulty  (p  =  .146  and  .926,  a  =  .05). 

•  The  user’s  perception  of  the  group’s  ability  to  stay  task-focused  (p  =  .829  and  .799,  a  =  .05). 

•  The  user’s  perception  of  productivity  in  terms  of  time  efficiency  (p  =  .807  and  .881,  a  =  .05). 

•  The  user’s  perception  of  productivity  in  terms  of  ability  to  reach  consensus  (p  =  .659  and 

1.000,  a  =  .05). 

•  The  user’s  perception  of  the  group’s  level  of  conflict  (p  =  .874  and  .842,  a  =  .05). 

•  The  user’s  perception  of  the  level  of  process  structure  (p  =  .580  and  .333,  a  =  .1  5). 

User  responses  recorded  on  the  post-process  questionnaires  are  summarized  in  tabular  form 
in  Appendix  J.  The  contingency  tables/Chi-Square  test  results  are  provided  in  Append :>  K,  and 
the  Fisher’s  Exact  test  results  are  shown  in  Appendix  L. 


Table  4.14.  Results:  Experiment  2,  Day  1  Participant  Responses — Chi-Square  Analysis. 


VARIABLE 

T  VALUE 

— 

X2  p  VALUE 

FISHER’S  EXACT 

p  VALUE 

DECISION 

Satisfied  with  Process 

0.529 

3.841 

0  467 

0.659 

Fail  to  Rej  *  H0 

Satisfied  with  Results 

3  232 

3  841 

0.072 

0.111 

Fail  to  Reject  H0 

Task  Difficulty 

7.917 

11  070 

0.161 

0.146 

Fail  to  Reject  H0 

Stay  Task- Focused 

0.558 

5  991 

0.757 

0.829 

Fail  to  Reject  H0 

Time  Efficiency 

1.190 

5  991 

0.551 

0.807 

Fail  to  Reject  Ho 

Reach  Consensus 

0.529 

3.841 

0.467 

0.659 

Fail  to  Reject  Ha 

Level  of  Conflict 

3  413 

11.070 

0.637 

0.874 

Fail  to  Reject  H0 

Level  of  Structure 

1.650 

5.991 

0.438 

0.580 

Fail  to  Reject  H0 
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Table  4.15.  Results:  Experiment  2,  Day  2  Participant  Responses — Chi-Square  Analysis. 


VARIABLE 

T  VALUE 

HH 

X2  p  VALUE 

FISHER’S  EXACT 

p  VALUE 

DECISION 

Satisfied  with  Process 

2.778 

7.815 

0.427 

0.654 

Fail  to  Reject  H0 

Satisfied  with  Results 

2  083 

7.815 

0.555 

0.884 

Fail  to  Reject  H0 

Task  Difficulty 

2.753 

11.070 

0.738 

0.926 

Fail  to  Reject  H0 

Stay  Task-Focused 

0.926 

5.991 

0.629 

0.799 

Fail  to  Reject  H0 

Time  Efficiency 

1.797 

7.815 

0.616 

0.881 

Fail  to  Reject  Ha 

Reach  Consensus 

0.926 

5.991 

0.629 

1.000 

Fail  to  Reject  H0 

Level  of  Conflict 

2.525 

9.488 

0.640 

0.842 

Psil  to  Reject  H0 

Level  of  Structure 

1,852 

5.991 

0.396 

0.333 

Fail  to  Reject  H0 

The  Sign  test  was  used  to  determine  if  the  FAC-MAN  treatment  yielded  higher  responses  for 
each  dependent  variable  of  interest  than  the  Manual  experimental  condition  for  those  groups  which 
experienced  both  experimental  conditions  over  the  two  days  (i.e.,  cross-over  Groups  5  and  6).  The 
Sign  test  compared  the  response  levels  (i.e.,  1-7)  by  participant  for  both  an  objective  test  and  a 
subjective  test.  The  objective  test  compared  the  two  responses  given  by  each  individual  for  every 
question  (dependent  variable)  from  one  experimental  condition  to  the  next.  The  subjective  test, 
however,  was  based  on  the  individual’s  own  personal  assessment  of  each  dependent  variable,  making 
a  comparison  between  the  two  experimental  conditions  at  the  same  time  (i.e.,  after  completion  of 
both  experimental  conditions).  The  hypotheses  for  both  the  objective  and  subjective  tests  were: 

H0  ■  p(+)  <  p(— )  or  FAC-MAN  <  Manual  for  the  Dependent  Variable  o]  Interest. 

Ha  '■  p(+)  >  p(-)  or  FAC-MAN  >  Manual  for  the  Dependent  Variable  of  Interest. 

Based  on  the  Sign  tests  which  evaluated  the  objective  tests,  the  null  hypothesis  was  rejected 
for  task  difficulty  ( p  =  .0000,  a  =  .05)  and  level  of  structure  (p=.0000,  o  =  .05)  as  shown  in 
Table  4.16;  however,  neither  of  these  null  hypotheses  was  rejected  under  the  subjective  test.  It 
was  concluded  that  the  facilitated  manual  process  was  perceived  as  being  more  difficult  and  more 
structured,  but  the  results  are  conflicting.  Sign  test  calculations  are  provided  in  Appendix  M. 

Based  on  the  Sign  tests  for  the  subjective  tests,  the  null  hypothesis  was  rejected  for  level  of 
conflict  (p  =  .0000,  a  =  .05)  as  shown  in  Table  4.16;  however,  the  null  hypothesis  for  this  dependent 
variable  was  not  rejected  under  the  objective  test.  It  was  concluded  that  the  facilitated  manual 
process  was  perceived  as  causing  greater  conflict,  but  the  results  are  conflicting. 

4.4.2  Discussion.  Even  though  some  of  the  contingency  tables  were  sparse,  it  was  interesting 
to  note  that  every  decision  concerning  rejection  of  the  null  hypothesis  (at  a  =  .05)  for  each  research 
objective  was  consistent  between  Fisher’s  Exact  test  and  the  Chi-Square  test. 
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Table  4.16.  Results:  Experiment  2,  Participant  Responses — Sign  Test  Analysis. 


OBJECTIVE 

SUBJECTIVE 

VARIABLE 

p  VALUE 

DECISION 

p  VALUE 

DECISION 

Satisfied  with  Process 

0.8125 

Fail  to  Reject  H0 

0.8125 

Sr.'  WPWSUMB 

Satisfied  with  Results 

0.9375 

Fail  to  Reject  H0 

0.9375 

Fail  to  Reject  Ha 

Task  Difficulty 

0.0000 

Reject  Ho 

0.2500 

Fail  to  Reject  Ha 

Stay  Task-Focused 

0.5000 

Fail  to  Reject  H0 

0.9375 

Fail  to  Reject  H0 

Time  Efficiency 

0.8750 

Fail  to  Reject  H0 

0.9844 

Fail  to  Reject  H0 

Reach  Consensus 

0.8906 

Fail  to  Reject  Ha 

0.9844 

Fail  to  Reject  H0 

Level  of  Conflict 

0.9687 

Fail  to  Reject  H0 

0.0000 

Reject  H0 

Level  of  Structure 

0.0000 

Reject  Ho 

0.1875 

Fail  to  Reject  Ha 

When  all  groups  were  considered  for  both  experiments,  there  were  no  statistically  significant 
differences  between  experimental  conditions  with  respect  to  the  user  responses.  However,  the  results 
from  evaluating  the  cross-over  groups  indicated  that  there  were  some  differences.  It  has  already 
been  shown  in  the  Sign  tests  comparing  these  cross-over  groups  that  the  objective  and  subjective 
tests  reached  conflicting  conclusions  for  a  few  of  the  dependent  variables  of  interest.  The  conclusions 
reached  for  one  dependent  variable  in  Experiment  1  and  three  dependent  variables  in  Experiment  2 
were  contradictory.  As  a  tertiary  objective  in  this  study,  the  reliability  of  the  questionnaire  as 
an  adequate  measuring  device  was  tested  also  using  the  Sign  test.  Here  a  comparison  was  made 
between  the  objective  and  subjective  results,  question  by  question,  for  the  responses  given  by  the 
cross-over  groups  for  both  experiments.  The  hypotheses  for  these  comparisons  were: 

Ho  :  p(+)  <  p(-)  or  Number  of  Times  Objective  and  Subjective  Results  Agree  <  Number  of  Times  Objective 
and  Subjective  Results  Disagree. 

Ha  ■  p(+)  >  p(~)  or  Number  of  Times  Objective  and  Subjective  Results  Agree  >  Number  of  Times  Objective 
and  Subjective  Results  Disagree. 

Based  on  the  results  from  Experiment  1  (reference  Table  4.17),  only  two  comparisons  resulted 
in  the  null  hypothesis  being  rejected:  satisfaction  with  the  process  (p  =  .0107,  a  =  .10)  and 
productivity  in  terms  of  time  efficiency  (p  =.0547,  a  =  .10).  It  was,  therefore,  concluded  that  only 
the  results  for  these  two  questions  in  the  Experiment  1  cross-over  groups  were  highly  reliable.  The 
Sign  test  calculations  are  provided  in  Appendix  M. 

Based  on  the  results  from  Experiment  2  (reference  Table  4.18),  four  out  of  the  eight  com¬ 
parisons  resulted  in  the  null  hypothesis  being  rejected:  satisfaction  with  the  process  (p  =  .0107, 
o  =  .10),  satisfaction  with  the  results  (p  =  .0107,  a  =  .10),  time  efficiency  (p  =.0547,  a  =  .10),  and 
level  of  structure  (p  =  .0547,  a  =  .10).  Although  level  of  structure  had  conflicting  objective  and 
subjective  Sign  test  results  (at  a  =  .05),  this  question  was  considered  highly  reliable  at  a  =  .10.  It 
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Table  4.17.  Results:  Experiment  1,  Questionnaire  Reliability — Sign  Test  Analysis. 


VARIABLE 

RELIABILITY 

p  VALUE  (a  =  .10) 

DECISION 

Satisfied  with  Process 

0.0107 

Reject  H0 

Satisfied  with  Results 

0.6230 

Fail  to  Reject  Ho 

ltak  Difficulty 

0.6230 

Fail  to  Reject  Ho 

Stay  Task- Focused 

0.8231 

Fail  to  Reject  Ho 

Time  Efficiency 

0.0547 

Reject  H0 

Reach  Consensus 

0.1719 

Fail  to  Reject  Ho 

Level  of  Conflict 

0.6230 

Fail  to  Reject  Ho 

Level  of  Structure 

0.8281 

Fail  to  Reject  Ho 

was  concluded  that  only  the  results  for  these  four  questions  (50%)  in  the  Experiment  2  cross-over 
groups  were  highly  reliable. 

Table  4.18.  Results:  Experiment  2,  Questionnaire  Reliability — Sign  Test  Analysis. 


VARIABLE 

RELIABILITY 

p  VALUE  (o  =  .10) 

DECISION 

Satisfied  with  Process 

0.0107 

Reject  Ho 

Satisfied  with  Results 

0.0107 

Reject  Ho 

Task  Difficulty 

0.1719 

Pail  to  Reject  H0 

Stay  Task-Focused 

0.1719 

Fail  to  Reject  Ho 

Time  Efficiency 

0.0547 

Reject  Hq 

Reach  Consensus 

0.1719 

Fail  to  Reject  Ho 

Level  of  Conflict 

0.9453 

Fail  to  Reject  Ho 

Level  of  Structure 

0.0547 

Reject  Ho 

4-5  Noteworthy  Facilitator  Comments 

All  facilitator  comments  provided  on  the  data  collection  forms  and  clarified  during  follow-up 
interviews  are  provided  in  Appendix  N.  Several  noteworthy  comments  are  provided  below. 

•  Five  of  the  six  facilitated  groups  (GDSS  and  FAC-MAN)  complained  that  the  environmental 
tasks  were  poorly  defined.  The  facilitators  for  a  couple  of  groups  noted  that  several  partic¬ 
ipants  required  clarification  of  the  training  levels  being  considered.  These  comments  were 
also  voiced  during  the  last  large  group  session  on  Day  2.  Since  these  definitions  were  pro¬ 
vided  by  the  sponsoring  organizations,  any  related  variability  could  not  be  controlled  by  the 
researcher.  Although  confusion  over  these  items  obviously  added  time  to  the  decision  making 
process  (i.e. ,  increased  decision  time)  and  potentially  affected  the  voting  process  (i.e.,  lower 
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consensus),  it  is  questionable  whether  they  confounded  the  study’s  results  concerning  the 
performance  outcomes,  since  all  groups  faced  the  same  conditions.  However,  they  may  have 
confounded  the  results  of  the  participants’  responses  on  the  post-process  questionnaires.  To 
be  specific,  while  some  participants  may  have  made  a  distinction  between  the  specific  decision 
making  process  (i.e.,  the  treatment)  versus  the  confusion  over  the  quality  of  the  information 
provided  by  the  sponsors,  others  may  not  have. 

•  One  facilitator  made  the  comment  that  the  use  of  a  GDSS  was  a  physical  barrier  to  effective 
communications.  He  noted  that  one  member  on  two  different  occasions  actually  stood  up  in 
order  to  get  his  point  across  to  the  other  group  members. 

•  The  two  GDSS  facilitators  noted  that  using  GroupSystems  allowed  the  groups  to  refer  to 
previously  cast  votes  to  ensure  consistency  in  the  needed  environmental  training  levels  across 
similar  career  fields.  Several  of  the  non-GDSS  facilitators  voiced  the  need  to  refer  to  previous 
votes.  There  was  a  clear  advantage  in  using  GDSS  to  recall  previously  accomplished  work. 

•  Some  groups  did  not  have  adequate  representation  for  some  career  fields.  Either  the  assigned 
representatives  were  not  true  experts  in  those  areas  or  there  was  actually  no  true  represen¬ 
tative  for  a  specific  career  field  assigned  to  the  group.  This  issue  may  have  increased  the 
decision  making  process  for  some  groups,  while  slowing  others  down.  No  conclusion  is  made 
concerning  the  effect  on  the  study’s  results.  Again,  this  area  could  not  be  controlled  by  the 
researcher,  since  attendance  at  the  conference  and  participant  assignments  to  groups  were 
handled  by  the  sponsors. 

•  The  researcher  and  the  facilitators  noted  that  some  groups  worked  well  together  while  others 
did  not.  Group  dynamics  appeared  to  influence  the  process  as  much  as  the  use  of  GDSS 
technology  or  facilitation.  Specifically,  Group  1  included  two  members  who  did  not  desire 
to  be  participants  in  the  small  group  processes.  One  technographer  noted  that  this  group 
had  “too  many  experts”  each  wanting  to  steer  the  decisions  of  the  group.  In  contrast,  Group 
3  worked  extremely  well  together.  The  facilitator  for  this  group  (both  days)  noted  that 
he  provided  little  facilitation  in  order  to  keep  the  group  moving— they  provided  their  own 
momentum.  It  was  interesting  to  note  these  observations  are  consistent  with  the  regression 
analysis  which  indicated  that  Group  3  was  the  fastest  group  in  Experiment  1  (p=.0014, 
a  =  .05). 
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4-6  Noteworthy  Participant  Comments 

Overwhelmingly,  the  participants  stated  that  the  environmental  tasks  and  training  levels  were 
poorly  defined.  Many  identified  that  the  representation  of  personnel  for  certain  career  fields  was 
inadequate  to  properly  assess  the  necessary  training  levels.  Except  for  these  areas,  there  was  no 
consensus  of  opinions  concerning  any  of  the  decision  making  processes.  Participant  comments  are 
provided  in  Appendix  O. 

Some  members  preferred  using  the  GDSS,  while  others  preferred  the  facilitated  manual  pro¬ 
cess.  For  example,  members  of  Groups  1,  2,  and  3  (i.e.,  groups  exposed  to  GDSS)  made  the 
following  comments: 

•  “I  liked  the  software  and  use  of  computers.” 

•  “Good  computer  program.  It  enhanced  our  performance.” 

•  “Due  to  use  of  the  computer,  [it  was]  easier  to  recall  [my]  own  position  and  the  group’s  overall 
position — helped  focus.” 

•  “[The]  computer  program  shows  other  career  field  scores.. .it  can  influence  your  decision.” 

•  “[GDSS  was]  grossly  time-consuming... no  flexibility.” 

•  “Computer  interferes  with  group  communications.” 

•  “Computer  process  work  was  too  long.” 

•  “For  small  groups,  this  [facilitated  manual]  system  is  preferred.  Larger  groups  would  benefit 
from  computer  software." 

Facilitation  techniques  were  also  either  praised  as  helping  the  process  or  discarded  as  unnec¬ 
essary  and  hindering  the  process  (Groups  1-6): 

•  “The  [GDSS]  facilitator  helped  by  repeating  the  task  several  times  with  the  type  of  worker 
keeping  focus  on  the  vote  at  hand.” 

•  “The  [GDSS]  facilitator  kept  us  focused  and  interacted  when  necessary  to  overcome  conflicts.” 

•  “Used  cards  (1-5)  [the]  first  day — [I]  liked  that  better  than  voice  voting.  Helped  in  reaching 
consensus.” 

•  Preferred  the  first  day’s  process  [FAC-MAN  over  the  Manual  process] — a  bit  more  time- 
consuming,  but  drives  more  individual  effort  prior  to  team  effort/concentration.” 

•  “Yesterday’s  session  had  a  facilitator  and  more  energetic  discussions,  but  also  seemed  more 
stressful.” 

•  “The  method  used  today  [FAC-MAN]  caused  more  conflicts  to  arise  and  the  process  was  less 
conducive  to  a  positive  result.” 

•  “The  ability  to  talk  through  conflict  and  reach  consensus  was  hampered  by  the  structure  of 
the  voting  process.  There  was  a  lower  level  of  interaction  among  group  members  today.  The 
formal  voting  system  didn’t  work  as  well.” 

Several  of  the  participants’  comments  were  repeated  in  the  large  group  session;  however,  none 
were  stated  as  the  consensus  of  the  whole  group.  One  original  comment,  though,  seemed  to  meet 
everyone’s  approval:  the  group  size  of  five  was  perfect,  providing  the  right  amount  of  breadth  of 
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experience  and  level  of  discussion.  Several  members  voiced  their  dislike  for  GDSS  or  for  a  structured 
facilitated  process,  while  others  opposed  both  of  these  views. 
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V.  Conclusion  and  Recommendations 


5.1  Overview  of  Ihe  Study 

The  current  research  contributed  to  the  study  of  Group  Decision  Support  Systems  in  field  ex¬ 
periments  which  involve  organizational  decision  making:  real  managers  making  real  world  decisions. 
An  independent  research  effort  conducted  by  the  Air  Force  Institute  of  Technology  in  cooperation 
with  the  Air  Force  Armstrong  Laboratory  evaluated  the  efficacy  of  using  GDSS  technology  in  an 
Air  Force  decision  making  environment. 

The  study  involved  middle  management  Air  Force  civilian  and  military  personnel  who  met 
in  small  five-member  groups  in  a  face-to-face  setting.  These  groups  participated  in  a  choice-task 
decision  making  process  over  a  two-day  period.  Each  group  attempted  to  reach  consensus  on 
200  decisions  concerning  environmental  training  levels  for  a  total  of  approximately  30  Air  Force 
professions. 

In  evaluating  the  success  of  GDSS  in  terms  of  efficiency  and  effectiveness,  past  laboratory 
studies  have  reached  very  contradictory  conclusions.  Recent  field  studies  have  proclaimed  fairly 
impressive  results;  however,  the  methods  used  in  these  studies  were  without  tight  experimental 
control,  and  the  comparisons  made  between  GDSS  and  non-GDSS  have  yielded  fairly  subjective 
results. 

The  current  research  provided  increased  experimental  control,  but  simultaneously  allowed 
enough  flexibility  to  perform  operational  decision  making  tasks.  This  study  objectively  compared 
facilitated  GDSS,  facilitated  manual,  and  unfacilitated  manual  groups  using  a  multi-methodological 
approach  by  evaluating  both  real-time  observations  of  the  group  decision  making  process  and  post- 
process  opinions  of  the  participants. 

The  research  evaluated  two  performance  outcomes:  the  efficiency  of  GDSS  in  terms  of  decision 
speed  and  the  effectiveness  of  GDSS  in  terms  of  consensus.  The  principle  focus  of  this  study  was 
to  determine  if  the  outcomes  obtained  were  the  result  of  using  a  GDSS  or  of  increased  process 
structure.  Consequently,  two  distinct  experiments  were  conducted.  The  participants  were  observed 
in  order  to  compare  each  group’s  decision  speed  for  20  distinct  intervals  and  each  group’s  consensus 
for  each  of  the  200  decisions.  Post-process  questionnaires  were  administered  to  measure  participant 
responses  and  compare  groups  with  respect  to  eight  secondary  outcomes: 

•  The  user’s  satisfaction  with  the  decision  making  process. 

•  The  user’s  satisfaction  with  the  group’s  results. 

•  The  user’s  perception  of  task  difficulty. 

•  The  user’s  perception  of  the  group’s  ability  to  remain  task-focused. 
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•  The  user’s  perception  of  the  group’s  productivity  in  terms  of  time  efficiency. 

•  The  user’s  perception  of  the  group’s  productivity  in  terms  of  the  group’s  ability  to  reach 
consensus. 

•  The  user’s  perception  of  the  group’s  level  of  conflict. 

•  The  user’s  perception  of  the  level  of  process  structure. 

Several  statistical  tests  were  performed  to  determine  which  of  the  outcomes  were  statistically 
significant  at  a  =  .05.  Multiple  regression  analysis  was  used  to  evaluate  Decision  Speed.  Contin¬ 
gency  tables/Chi-Square  tests  were  used  to  evaluate  Consensus.  Contingency  tables/Chi-Square 
and  Sign  tests  were  used  to  evaluate  participant’s  responses  for  the  secondary  outcomes.  Finally, 
the  Sign  test  was  used  to  evaluate  the  reliability  of  the  post-process  questionnaire  at  a  =  .10. 


5.2  Significant  Findings  for  the  Performance  Outcomes 


5.2.1  Decision  Speed. 

5. 2. 1.1  Experiment  1:  GDSS  Effects  There  was  a  significant  relationship  between  the 
facilitated  GDSS  process  and  decision  speed  (p  =  .0096,  a  =  .05).  It  took  facilitated  GDSS  groups 
longer  to  complete  the  decision  making  tasks  than  facilitated  manual  groups. 

5. 2. 1.2  Experiment  2:  Structure  Effects.  There  was  a  significant  relationship  between 
the  facilitated  manual  process  and  decision  speed  (p  =  .0265,  a  =  .05).  It  took  facilitated  manual 
groups  longer  to  complete  the  decision  making  tasks  than  unfacilitated  manual  groups. 


5.2.2  Consensus. 

5.2.2. 1  Experiment  1:  GDSS  Effects.  The  level  of  consensus,  in  terms  of  the  total 
number  of  tasks  for  which  the  groups  reached  consensus,  between  facilitated  GDSS  groups  and 
facilitated  manual  groups  was  not  the  same  (p  =  .042,  a  =  .05).  The  level  of  consensus  in  facilitated 
GDSS  groups  was  proportionally  lower  than  the  level  of  consensus  in  facilitated  manual  groups. 

The  ability  to  reach  consensus  between  facilitated  GDSS  and  facilitated  manual  groups  was 
not  the  same  (p  =  .0006,  a  =  .05).  Proportionally,  facilitated  manual  groups  reached  consensus  in 
fewer  votes  than  did  facilitated  GDSS  groups. 

5. 2. 2. 2  Experiment  2:  Structure  Effects.  The  level  of  consensus  between  facilitated 
manual  and  unfacilitated  manual  groups  was  not  the  same  (p  =  .0006,  a  =  .05).  The  level  of 


5-2 


consensus  in  facilitated  manual  groups  was  proportionally  lower  than  the  level  of  consensus  in 
unfacilitated  manual  groups. 

5.2.3  Concluding  Remarks  Concerning  Performance  Outcomes.  It  was  noted  that  the  struc¬ 
tured  processes  exhibited  greater  initial  learning  curves  for  one  or  two  intervals  whenever  groups 
transitioned  from  a  less  structured  process  to  a  more  structured  process.  Specifically,  facilitated 
GDSS  groups  experienced  a  large  learning  curve,  facilitated  manual  groups  experienced  a  medium 
learning  curve,  and  unfacilitated  manual  groups  experienced  a  small  learning  curve.  Groups  tran¬ 
sitioning  in  the  opposite  direction  did  not  experience  such  an  increase. 

Facilitated  GDSS  groups  experienced  slower  decision  speeds  than  facilitated  manual  groups 
which,  in  turn,  experienced  slower  decision  speeds  than  that  of  unfacilitated  manual  groups.  From 
Experiment  2  there  was  a  clear  indication  that  process  structure  did  adversely  affect  decision  speed; 
however,  group  dynamics  probably  contributed  to  the  differences  experienced  in  the  facilitated 
manual  and  facilitated  GDSS  groups  in  Experiment  1.  Based  on  the  facilitator  and  participant 
comments,  groups  which  seemed  to  work  well  together  tended  to  produce  better  results.  Finally, 
decision  speed  generally  improved  for  all  groups  as  time  progressed.  This  observation  supported 
the  theory  of  the  learning  curve  effect. 

Proportionally,  facilitated  GDSS  groups  reached  consensus  less  frequently  than  did  facilitated 
manual  groups  which,  in  turn,  reached  consensus  less  frequently  (i.e.,  proportionally)  than  did 
unfacilitated  manual  groups.  It  also  took  facilitated  GDSS  groups  more  votes  proportionally  to 
reach  consensus  than  it  did  for  facilitated  manual  groups.  These  observations  suggest  that  structure 
adversely  affects  the  rate  at  which  groups  achieve  consensus;  however,  group  dynamics  probably 
contributed  to  the  significant  differences  in  the  outcomes  of  the  facilitated  GDSS  and  facilitated 
manual  groups. 


5.3  Significant  Findings  for  the  Secondary  Outcomes 

5.3.1  Experiment  1,  Groups  1~4:  GDSS  Effects.  Considering  all  Experiment  1  groups  for 
both  days,  there  was  no  difference  between  facilitated  GDSS  and  facilitated  manual  groups  with 
respect  to  the  following  eight  secondary  outcomes: 

•  The  user’s  satisfaction  with  the  decision  making  process  (p  =  .288  and  .614,  a  =  .05). 

•  The  user’s  satisfaction  with  the  results  of  the  decision  making  process  (p  =  1.000  and  .888, 
a  =  .05). 

•  The  user’s  perception  of  task  difficulty  (p  =  .972  and  .907,  a  =  .05). 

•  The  user’s  perception  of  the  group’s  ability  to  stay  task-focused  (p  =  .811  and  .799,  a  =  .05). 
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•  The  user’s  perception  of  productivity  in  terms  of  time  efficiency  (p  =  .097  and  .890,  a  =  .05). 

•  The  user’s  perception  of  productivity  in  terms  of  ability  to  reach  consensus  (p  =  .317  and 
.460,  a  =  .05). 

•  The  user’s  perception  of  the  group’s  level  of  conflict  (p  =  .932  and  .791,  a  =  .05). 

•  The  user’s  perception  of  the  level  of  process  structure  (p  =  1.000  and  .835,  a  =  .05). 

5.S.S  Experiment  1,  Groups  1  &  2:  GDSS  Effects.  Specifically  evaluating  only  the  cross¬ 
over  groups,  facilitated  GDSS  groups  did  not  experience  higher  levels  of  response  for  any  of  the  eight 
secondary  outcomes  for  the  objective  tests  which  compared  participant’s  responses  between  exper¬ 
imental  conditions.  For  the  subjective  tests  which  required  the  participant  to  make  a  comparison, 
facilitated  GDSS  groups  perceived  the  task  as  being  more  difficult  (p  =  .0156,  a  =  .05). 

5.3.3  Experiment  2,  Groups  \-7:  Structure  Effects.  Considering  all  Experiment  2  groups 
for  both  days,  there  was  no  difference  between  facilitated  manual  and  unfacilitated  manual  groups 
with  respect  to  the  following  eight  secondary  outcomes: 

•  The  user’s  satisfaction  with  the  decision  making  process  (p  =  .659  and  .654,  a  =  .05). 

•  The  user’s  satisfaction  with  the  results  of  the  decision  making  process  (p  =  .111  and  .884, 
o  =  .05). 

•  The  user’s  perception  of  task  difficulty  (p  =  .146  and  .926,  o  =  .05). 

•  The  user’s  perception  of  the  group’s  ability  to  stay  task-focused  (p  =  .829  and  .799,  a  =  .05). 

•  The  user’s  perception  of  productivity  in  terms  of  time  efficiency  (p  =  .807  and  .881,  a  =  .05). 

•  The  user’s  perception  of  productivity  in  terms  of  ability  to  reach  consensus  (p  =  .659  and 
1.000,  o  =  .05). 

•  The  user’s  perception  of  the  group’s  level  of  conflict  (p  =  .874  and  .842,  a  =  .05). 

•  The  user's  perception  of  the  level  of  process  structure  (p  =  .580  and  .333,  a  =  .05). 

5.3.4  Experiment  2,  Groups  5  &  6:  Structure  Effects.  Specifically  evaluating  only  the  cross¬ 
over  groups,  facilitated  manual  groups  perceived  tasks  to  be  more  difficult  (p  =  .0000,  a  =  .05) 
and  more  structured  (p=.0000,  o  =  .05)  for  the  objective  tests  which  compared  participant’s 
responses  between  experimental  conditions.  For  the  subjective  tests  which  required  the  participant 
to  make  a  comparison,  facilitated  manual  groups  perceived  a  higher  level  of  group  conflict  than  the 
unfacilitated  manual  groups  (p  =  .0000,  a  =  .05). 

5.3.5  Concluding  Remarks  Concerning  Secondary  Outcomes.  Whenever  all  groups  in  Exper¬ 
iment  1  were  taken  into  consideration,  none  of  the  secondary  outcomes  were  considered  significant. 
Considering  only  the  cross-over  groups,  however,  facilitated  GDSS  groups  perceived  the  task  to  be 
more  difficult,  but  the  objective  and  subjective  results  conflict.  It  was  concluded  that  facilitated 
GDSS  participants  perceived  the  process  as  being  more  difficult. 
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The  same  situation  occurred  in  Experiment  2.  None  of  the  secondary  outcomes  were  consid¬ 
ered  significant  when  all  groups  were  compared;  however,  the  cross-over  groups  perceived  a  higher 
level  of  task  difficulty,  a  higher  level  of  structure,  and  a  higher  level  of  conflict  for  the  facilitated 
manual  process.  Again,  the  objective  and  subjective  test  results  conflict.  It  was  concluded  that 
facilitated  manual  participants  perceived  higher  levels  of  task  difficulty,  structure,  and  conflict. 

5.3.6  Questionnaire  Reliability.  As  was  shown  in  the  objective  and  subjective  tests  of  the 
secondary  outcomes,  the  conclusions  reached  for  one  dependent  variable  in  Experiment  1  and 
three  dependent  variables  in  Experiment  2  were  contradictory.  A  comparison  of  the  objective  and 
subjective  results  for  Experiment  1  indicated  that  only  two  of  the  eight  secondary  outcomes  were 
highly  reliable:  user  satisfaction  with  the  process  ( p  =  .0107,  a  =  .10)  and  productivity  in  terms 
of  time  efficiency  (p  =.0547,  a  =  .10).  A  similar  comparison  for  Experiment  2  indicated  that  only 
four  of  the  eight  secondary  outcomes  were  highly  reliable:  user  satisfaction  with  the  process  (p  = 
.0107,  a  =  .10),  user  satisfaction  with  the  results  (p  =  .0107,  a  =  .10).  user  perception  of  time 
efficiency  (p  =.0547,  a  =  .10),  and  user  perception  of  the  level  of  structure  (p  =  .0547,  a  =  .10). 

5.4  Conclusions  Based  on  Facilitator  and  Participant  Comments 

Five  of  the  six  facilitated  groups  stated  that  the  environmental  tasks  were  not  adequately 
defined.  Also,  several  participants  in  various  groups  stated  that  there  was  not  adequate  repre¬ 
sentation  for  some  career  fields  in  their  groups.  Since  these  comments  were  randomly  scattered 
throughout  the  groups,  any  adverse  effects  should  have  been  experienced  across  most  all  groups. 
However,  no  precise  conclusion  could  be  made  concerning  the  effect  on  the  study’s  results. 

Several  participants  felt  that  use  of  a  GDSS  was  time-consuming,  and  that  it  physically 
interfered  with  effective  group  communications.  On  the  contrary,  other  members  praised  the  GDSS, 
because  it  provided  the  capability  to  recall  previously  cast  votes  which  helped  groups  maintain 
consistency  in  the  assignment  of  environmental  training  levels  for  similar  career  fields.  Support  and 
non-support  of  the  GDSS  seemed  to  be  group  dependent — again  an  indication  of  the  contributing 
factors  associated  with  group  dynamics. 

The  comments  concerning  facilitation  were  also  mixed.  Some  members  claimed  that  facilita¬ 
tion  helped  keep  the  group  focused,  but  it  required  a  greater  level  of  concentration.  Some  members 
simply  preferred  the  unfacilitated  manual  method. 
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One  final  comment  made  during  the  large  group  session  appeared  to  meet  consensus  across  all 
the  groups.  They  stated  that  the  group  size  of  five  members  was  ideal,  providing  the  right  amount 
of  breadth  of  experience  and  level  of  discussion. 

5.5  Conclusions  Applicable  to  Previous  GDSS  Field  Studies 

Although  the  results  stemming  from  the  earlier  laboratory  studies  have  been  largely  con¬ 
tradictory,  the  field  studies  have  resulted  in  several  fairly  consistent  findings  with  respect  to  the 
success  or  failure  of  the  application  of  GDSS  (or  GSS)  in  real  world  environments.  Several  general 
findings  are  repeated  here  for  comparison  purposes. 

•  GDSS  “technology  used  in  the  field  appears  to  improve  performance,  efficiency,  and  user 
satisfaction”  (6:73),  (22:219).  This  study  contradicts  these  findings,  since  performance,  in 
terms  of  consensus,  and  efficiency  were  shown  to  be  lower  for  facilitated  GDSS  groups  than 
both  facilitated  manual  and  unfacilitated  manual  groups.  This  study  found  no  difference  in 
the  level  of  satisfaction  for  any  of  the  three  experimental  conditions. 

•  “The  effects  of  GSS  appear  to  depend  on  a  variety  of  factors:  the  technology  itself,  the  group, 
and  the  task”  (6:73).  This  study  essentially  supports  these  findings,  especially  since  group 
dynamics  appeared  to  be  as  much  a  contributing  factor  to  the  results  as  the  technology, 
specifically  with  respect  to  the  secondary  outcomes. 

•  “We  are  less  convinced  that  GSS  technology  can  help  groups  facing  a  choice  task,  where  the 
objective  is  to  choose  an  alternative(s)  from  a  pre-specified  set.  For  these  types  of  tasks,  GSS 
technology  may  help,  but  the  evidence  is  not  as  clear  as  that  for  generation  tasks”  (6:74). 
This  study  clearly  confirms  this  finding,  although  group  dynamics  was  a  contributing  factor 
to  the  results. 

•  “There  is  better  fit  between  GSS  and  complex  tasks”  (6:74).  This  study  neither  confirms 
nor  contradicts  this  finding  with  respect  to  complex  tasks;  however,  the  study  does  confirm 
that  GDSS  use  did  not  result  in  improved  performance  for  relatively  simple  tasks  such  as 
choice-type  decision  making. 

•  “Efficiency  considerations  of  GDSS  become  increasingly  apparent  as  group  size  increases. 
It  is  difficult  to  demonstrate  that  GDSS  promotes  group  efficiency  for  small  groups  (e.g., 
3  to  5). ..Group  effectiveness  is  enhanced  as  group  size  increases. ..the  effectiveness  of  GDSS 
becomes  apparent  in  facilitating  large  numbers  of  issues  associated  with  a  complex  question” 
(41:124).  Generally  speaking,  this  study  confirms  that  the  efficiency  and  effectiveness  for 
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small  groups  performing  a  relatively  simple  choice-task  using  GDSS  were  not  better  than  the 
efficiency  and  effectiveness  of  non-GDSS  groups. 

•  “The  value  of  process  structure  for  larger  groups  has  been  demonstrated  in  numerous  field 
studies... process  structure  is  useful  for  large  groups  facing  complex  tasks,  but  may  not  have 
much  effect  on  small  groups  or  groups  performing  well-defined  tasks”  (31:138).  The  results  of 
this  study  contradict  these  findings.  Process  structure  did  have  an  effect  on  the  small  groups 
in  this  study;  however,  the  effect  negatively  impacted  the  performance  outcomes. 


5.6  Recommendations 

It  is  certainly  too  early  in  the  development  and  study  of  GDSS  for  the  Air  Force  to  make 
significant  investments  in  this  technology.  Armstrong  Laboratory  and  the  Air  Force  Institute  of 
Technology  should  continue  the  partnership  to  conduct  objective  field  research.  This  research 
is  needed  to  adequately  determine  which  types  of  Air  Force  groups  and  what  types  of  Air  Force 
decision  making  tasks  are  best  suited  for  GDSS  application.  Additionally,  further  study  is  warranted 
to  identify  decision  quality  variables  which  constitute  appropriate  measures  of  effectiveness  for  a 
variety  of  Air  Force  decision  making  tasks.  Based  on  the  results  of  this  study  and  the  findings  from 
other  field  studies,  the  following  additional  research  is  recommended: 

•  Conduct  a  factorial  experiment  in  a  similar  choice-type  decision  making  process  to  study  two 
factors:  group  size  (i.e. ,  groups  of  5  versus  10  members)  and  technology  (i.e.,  GDSS  versus 
non-GDSS).  Such  a  study  would  attempt  to  validate  the  finding  that  large  groups  benefit 
from  GDSS  use  more  than  small  groups,  in  addition  to  studying  the  presence  or  absence  of 
GDSS  technology. 

•  Conduct  a  study  similar  to  the  current  study,  except  increase  the  complexity  of  the  task  by 
having  members  generate  and  rank  order  alternatives  and  then  choose  a  course  of  action. 
Such  a  study  would  attempt  to  validate  the  finding  that  more  complex  tasks  benefit  from 
GDSS  use. 

•  Conduct  a  longitudinal  study  to  determine  specific  functional  areas  and  types  of  tasks  that 
may  best  benefit  from  use  of  GDSS.  Such  a  study  should  place  particular  attention  on  eval¬ 
uating  the  effects  of  group  dynamics  to  the  outcomes  of  the  decision  making  processes. 

•  Eventually,  conduct  a  factorial  experiment  to  study  two  factors:  group  size  (i.e.,  5  or  10 
members  per  group)  and  group  proximity  (i.e.,  face-to-face  versus  dispersed).  This  is  one 
area  where  Air  Force  decision  makers  may  benefit  fiscally,  should  the  technology  produce 
positive  results  and  be  accepted  by  Air  Force  members. 
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•  Conduct  a  longitudinal  study  to  develop  and  validate  a  reliable  survey  instrument  to  ac¬ 
curately  measure  the  perceptions  of  GDSS  users.  It  has  already  been  shown  that  existing 
instruments  produce  somewhat  unreliable  results.  Such  instruments  will  be  necessary  to  cap¬ 
ture  users’  acceptance  of  the  technology  prior  to  fielding  operational  systems  in  the  work 
place. 

Comments  from  this  study  identified  a  genuine  need  to  reduce  the  physical  obstruction  of  the 
GDSS  hardware.  These  limitations  adversely  affected  communications  between  group  members. 
Recommend  that  Armstrong  Laboratory  invest  in  computer  consoles  which  recess  the  computer 
and  its  monitor  out  of  the  direct  line-of-sight  between  group  members.  This  will  preclude  the 
technology  from  directly  interfering  with  effective  group  communications.  Also,  recommend  that 
Armstrong  Laboratory  acquire  an  audio  recording  capability  so  that  group  conversations  may 
be  captured  and  later  analyzed.  This  capability  will  be  essential  towards  evaluating  the  group 
dynamics  aspects  of  GDSS  research. 
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Appendix  A.  Post-Process  Questionnaire:  Day  One 


Post-Process  Questionnaire 


Page  1  of  3 


Group/lndividual  Number: _  Date: _ 

Please  answer  the  following  questions  using  the  scale  provided  with  each  respective  question. 
Please  circle  only  ONE  number  which  best  describes  your  opinion  for  each  particular  question. 


1 .  How  satisfied  are  you  with  the  RESULTS  your  group  produced? 


i — i — ! — i — t— i 


2.  Taking  all  things  into  consideration,  how  DIFFICULT  was  this  group  task? 


I-  h  111) 


3.  To  what  degree  was  your  group  able  to  STAY  FOCUSED  on  completing  the  task? 


i — I— 1— I — h— I 


4.  In  terms  of  TIME  EFFICIENCY,  how  productive  was  your  group? 


MeMra*  Saar 


unpeauaiM  Mm  wayun  emumta  wane 

i — 1 — 1 — I — I — I 


5.  In  terms  of  agreement/disagreement,  what  level  of  CONFLICT  did  your  group  experience? 


Emm*  M< 


wiwim  IMew*  Pi^eni  PMarM  /tqmmtm  A*mm»  »|him 

(— 1 — i — t — t — I — I 


6.  In  terms  of  ABILITY  TO  REACH  CONSENSUS,  how  productive  was  your  group? 


(continued  on  next  page) 


Post-Process  Questionnaire 
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7.  Overall,  how  satisfied  are  you  with  the  GROUP  PROCESS  you  just  completed? 


8.  Based  on  your  perceptions,  how  STRUCTURED  was  this  group  process? 


fiww* 


9.  Please  provide  any  comments  concerning  POSITIVE  aspects  of  your  group's  process 
or  experience. 


1 0.  Please  provide  any  comments  concerning  NEGATIVE  aspects  of  your  group's  process 
or  experience. 


(continued  on  next  page) 
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Post-Process  Questionnaire 


1 1 .  Please  provide  any  additional  comments  below. 


Appendix  B.  Post-Process  Questionnaire:  Day  Two 


\ 
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Post-Process  Questionnaire 


Page  1  of  3 


Group/Individual  Number: _  Date: _ 

Please  answer  the  following  questions  using  the  scale  provided  with  each  respective  question. 
Please  circle  only  ONE  number  which  best  describes  your  opinion  for  each  particular  question. 

1 .  How  satisfied  are  you  with  the  RESULTS  your  group  produced? 


2.  Taking  all  things  into  consideration,  how  DIFFICULT  was  this  group  task? 


3.  To  what  degree  was  your  group  able  to  STAY  FOCUSED  on  completing  the  task? 


4.  In  terms  of  TIME  EFFICIENCY,  how  productive  was  your  group? 


5.  In  terms  of  agreement/disagreement,  what  level  of  CONFLICT  did  your  group  experience? 


6.  In  terms  of  ABILITY  TO  REACH  CONSENSUS,  how  productive  was  your  group? 


(continued  on  next  page) 
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7.  Overall,  how  satisfied  are  you  with  the  GROUP  PROCESS  you  just  completed? 


8.  Based  on  your  perceptions,  how  STRUCTURED  was  this  group  process? 


9.  Comparing  today's  small  group  process  with  yesterday  afternoon's  small  group  process, 
how  would  you  rate  the  following  items : 


TM* 

Unchanged 

YaamUy 

a.  Your  level  of  satisfaction  with  the  group's  outcome? 

□ 

□ 

□ 

b.  Your  perception  of  the  level  of  task  difficulty? 

□ 

□ 

□ 

c.  Your  group's  ability  to  remain  focused  on  the  task? 

□ 

□ 

□ 

d.  Your  group's  overall  productivity  in  terms  of  time  efficiency? 

□ 

□ 

□ 

e  Your  perception  of  the  level  of  conflict  within  the  group? 

□ 

□ 

□ 

f .  Your  group’s  overall  productivity  in  terms  of  ability  to  reach  consensus? 

□ 

□ 

□ 

g.  Your  level  of  satisfaction  with  the  group  process? 

□ 

□ 

□ 

h.  Your  perception  of  the  level  of  group  process  structure? 

□ 

□ 

□ 

10.  Please  provide  any  comments  concerning  POSITIVE  aspects  of  your  group's  process  or  experience. 


(continued  on  next  page) 
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1 1 .  Please  provide  any  comments  concerning  NEGATIVE  aspects  of  your  group's  process 
or  experience. 


12.  Please  provide  specific  reasons  why  you  may  have  preferred  one  day’s  group 
process  over  the  other  day's  (or  any  other  comments  you  wish  to  make). 


Appendix  C.  Participant  Demographics:  Pre-Process  Questionnaire  Results 


Appendix  D.  Data  Correlation  and  Equations  for  Experiment  1 
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CORRELATIOI  BETVEEI  SI  AID  EM 
IITERVAL  1  TO  10 


Correlation  Analysis 


2  ’VAR'  Variables:  SI  EM 


Simple  Statistics 


Variable 

I 

Mean 

Std  Dev 

Sum 

Minimum 

SI 

10 

14.3833 

9.9233 

143.8330 

4.7600 

EM 

10 

13.2368 

10.9805 

132.3E80 

5.6000 

Pearson 

Correlation  Coefficients  /  Prob  > 

|R|  under  Ho: 

Rho=0  /  I 

SI 

EM 

SI 

1.00000 

0.96371 

0.0 

0.0001 

EM 

0.96371 

1.00000 

0.0001 

0.0 

Mathematica  Equations  for  SI  and  EM  during  Intervals  1-10 


Equation  for  SI: 

y  =  90.0082  -  73.3668*  +  27.9537xJ  -  5.2460*3  +  0.472959*4  -  0.0162701*8 
Equation  for  EM: 

y  =  75.1361  -44.4634*+  11.7412*2-  1.53825*3  +  0.10042*4  -  0.00264731*8 


D-2 


Maximum 

39.S830 

40.4160 

=  10 


Appendix  E.  Decision  Speed  Interval  Data 
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E-2 


Table  E.3.  Interval  Times  for  Experiment  2,  Groups  4  and  5 


GROUP 

DAY 

INTERVAL 

FACILITATOR 

TIME  (ntlia) 

4 

1 

1 

YES 

SI 

39.663 

4 

1 

2 

YES 

SI 

31.167 

4 

1 

3 

YES 

SI 

ia.75 

4 

1 

4 

YES 

SI 

13.167 

4 

1 

5 

YES 

SI 

14.063 

4 

1 

6 

YES 

SI 

4.75 

4 

1 

7 

YES 

SI 

12.25 

4 

1 

8 

YES 

SI 

7.T5 

4 

1 

9 

YES 

SI 

13.063 

4 

1 

10 

YES 

SI 

7.25 

4 

a 

11 

YES 

EM 

17.063 

4 

3 

la 

YES 

EM 

15.167 

4 

2 

13 

YES 

EM 

16.633 

4 

2 

14 

YES 

EM 

20.7 

4 

2 

15 

YES 

EM 

11.3 

4 

2 

16 

YES 

EM 

4 

2 

17 

YES 

EM 

4 

2 

16 

YES 

EM 

4 

2 

19 

YES 

EM 

4 

2 

ao 

YES 

EM 

5 

1 

l 

YES 

EM 

40.416 

6 

1 

a 

YES 

EM 

34.633 

5 

1 

3 

YES 

EM 

9.667 

5 

1 

4 

YES 

EM 

11.5 

5 

1 

5 

YES 

EM 

6.95 

5 

1 

6 

YES 

EM 

6.16  | 

5 

1 

7 

YES 

EM 

6.5 

s 

1 

8 

YES 

EM 

6.416 

5 

1 

9 

YES 

EM 

6.416 

5 

1 

10 

YES 

EM 

5.5 

5 

2 

11 

NO 

NONE 

6.063 

5 

2 

13 

NO 

NONE 

9.63 

5 

2 

13 

NO 

NONE 

10.933 

5 

2 

14 

NO 

NONE 

5.066 

5 

2 

15 

NO 

NONE 

6.3 

5 

2 

16 

NO 

NONE 

11.75 

5 

2 

17 

NO 

NONE 

7.45 

5 

2 

16 

NO 

NONE 

10.433 

5 

mm 

19 

NO 

NONE 

6.ai7 

S 

mm 

ao 

NO 

NONE 

4.167 

E-4 


Table  E.4.  Interval  Times  for  Experiment  2,  Groups  6  and  7 


Appendix  F.  Regression  Analysis  Results  for  Decision  Speed 


F-l 


EXPERIMENT  1:  GDSS  VERSUS  FACILITATED-MAHUAL 
REGRESSION  AIALYSIS  USIIG  I1DICATOR  VARIABLES 


Model:  MODEL1 
Dependent  Variable:  TIME 


Analysis  o 1  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Model 

6 

2687.18974 

447.86496 

13.486 

Error 

53 

1760.08047 

33.20907 

C  Total 

59 

4447.27020 

Root  MSE  6. 76273  R-sqnare  0.6042 
Dep  Mean  13.21183  Adj  R-sq  0.5594 
C.V.  43.61795 


Parameter  Estimates 


Prob>F 

0.0001 


Parameter 

Standard 

T  for  HO: 

Variable 

DF 

Estimate 

Error 

Parameter=0 

Prob  >  |T| 

IHTERCEP 

1 

24.070314 

2.16227846 

11.132 

0.0001 

GROUP 1 

1 

-0.418325 

2.37244192 

-0.176 

0.8607 

GR0UP2 

1 

-2.044866 

2.79508739 

-0.732 

0.4676 

GR0UP3 

1 

-10.976979 

3.42756253 

-3.203 

0.0023 

DAY 

1 

-4.388051 

1.57141794 

-2.792 

0.0073 

INTERVAL 

1 

-1.631407 

0.29457261 

-5.638 

0.0001 

FACILITY 

1 

7.465827 

2.63585370 

2.832 

0.0065 

Variance 

Variable 

DF 

Inflation 

IHTERCEP 

1 

0.00000000 

GROUP 1 

1 

1.90672055 

GROUP 2 

1 

1.96043696 

GR0UP3 

1 

4.71685858 

DAY 

1 

1.11536758 

INTERVAL 

1 

1.26618320 

FACILITY 

1 

3.13817538 

Durbin-Vatson  D  1.933 
(For  Humber  of  Obs . )  60 
1st  Order  Autocorrelation  0.032 


F-2 


Dep  Var 

Predict 

Obs 

TINE 

Value 

1 

32.0000 

29.4864 

2 

40.6700 

27.8550 

3 

16.5800 

26.2236 

4 

30.2500 

24.5922 

5 

16.3300 

22.9608 

6 

12.6600 

17.6325 

7 

13.1000 

16.0011 

8 

14.7500 

14.3697 

9 

13.9300 

12.7383 

10 

5.4600 

11.1069 

11 

6.0330 

9.4755 

12 

7.5500 

7.8441 

13 

6.2500 

6.2127 

14 

7.5170 

4.5813 

16 

10.9500 

2.9499 

16 

21.3500 

12.2370 

17 

9.6670 

10.6056 

18 

8.3330 

8.9742 

19 

6.6670 

7.3428 

20 

3.5660 

6.7114 

21 

41.9170 

23.4718 

22 

16.8330 

21.8404 

23 

10.3330 

20.2090 

24 

13.6670 

18.5776 

25 

13.5830 

16.9462 

26 

20.8500 

18.9278 

27 

16.6670 

17.2963 

28 

11.3830 

15.6649 

29 

13.7500 

14.0335 

30 

11.6830 

12.4021 

31 

7.6500 

10.7707 

32 

12.1500 

9.1393 

33 

6.7830 

7.5079 

34 

5.7000 

5.8765 

35 

6.9670 

4.2451 

36 

11.5000 

14.5397 

37 

13.7330 

12.9083 

38 

7.6000 

11.2769 

39 

5.0670 

9.6456 

40 

12.5830 

8.0141 

41 

7.2000 

6.3827 

42 

3.8330 

4.7513 

43 

6.6500 

3.1199 

44 

3.1830 

1.4885 

45 

2.9160 

-0.1430 

46 

39 . 5830 

22.4389 

F-3 


Residual 

2.5136 
12.8150 
-9.6436 
S.6578 
-6.6308 
-4.9725 
-2.9011 
0.3803 
1.1917 
-5.6469 
-3.4425 
-0 . 2941 
0.0373 
2.9357 
8.0001 
9.1130 
-0.9386 
-0.6412 
-0.6758 
-2.1454 
18.4452 
-5.0074 
-9.8760 
-4.9106 
-3.3632 
1.9222 
-0.6293 
-4.2819 
-0.2835 
-0.7191 
-3.1207 
3.0107 
-0.7249 
-0.1765 
2.7219 
-3.0397 
0.8247 
-3.6769 
-4.5785 
4.5689 
0.8173 
-0.9183 
3.5301 
1.6945 
3.0590 
17.1441 


47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

Sum  of  Residuals 

Stun  of  Squared  Residuals 

Predicted  Resid  SS  (Press) 


21.1670 

20.8075 

0.3595 

12.7600 

19.1761 

-6.4261 

12.1670 

17.5447 

-5.3777 

14.0830 

15.9133 

-1.8303 

4.7500 

14.2819 

-9.5319 

12.2500 

12.6505 

-0.4005 

7.7500 

11.0191 

-3.2691 

12.0830 

9 . 3877 

2.6953 

7.2500 

7.7562 

-0.5062 

17.0830 

18.0509 

-0.9679 

15.1670 

16.4194 

-1.2524 

16.8330 

14.7880 

2.0450 

20.7000 

13.1566 

7.5434 

11.3000 

11.5252 

-0.2252 

0 

1760.0805 

2394.8191 


F-4 


- + - + - + - + - + - + - + - + - + - + - 4 - + - + - 


RESIDUAL  I 
20  + 


I 

+ 


15  + 


10  + 


R 

e 

s 

i 

d 

u 

a 

1 


5  + 


0  + 


-5  + 


*  **  ** 

*  * 

*  * 

*  *  * 

*  *  *  * 
******  ** 

* 

*  * 

*  *  * 

*  **  * 

*  * 

*  * 

*  * 


-10  + 


— + - + - + - + - + - + - + - + - + - + - + - + - +— 

0.0  2.5  5.0  7.5  10.0  12.5  15.0  17.5  20.0  22.5  25.0  27.5  30.0 

Predicted  Value  of  TIKE  PEED 


F-5 


Dep  Var 

Predict 

Std  Err 

Std  Err 

Student 

Obs 

TIME 

Value 

Predict 

Residual 

Residual 

Residual 

1 

32.0000 

29.4864 

2.415 

2.5136 

5.232 

0.480 

2 

40.6700 

27.8550 

2.364 

12.8150 

5.256 

2.438 

3 

16.5800 

26.2236 

2.349 

-9.6436 

5.262 

-1.833 

4 

30.2500 

24.5922 

2.370 

5.6578 

5.253 

1.077 

5 

16.3300 

22.9608 

2.428 

-6.6308 

5.226 

-1.269 

6 

12.6600 

17.6325 

2.198 

-4.9725 

6.327 

-0.933 

7 

13.1000 

16.0011 

2.032 

-2.9011 

5.393 

-0.538 

8 

14.7500 

14.3697 

1.897 

0.3803 

5.441 

0.070 

9 

13.9300 

12.7383 

1.802 

1.1917 

5.474 

0.218 

10 

5.4600 

11.1069 

1.751 

-5.6469 

5.490 

-1.029 

11 

6.0330 

9.4765 

1.749 

-3.4425 

5.491 

-0.627 

12 

7.6600 

7.8441 

1.796 

-0.2941 

5.476 

-0.054 

13 

6.2500 

6.2127 

1.888 

0.0373 

5.445 

0.007 

14 

7.5170 

4.5813 

2.019 

2.9357 

5.397 

0.544 

15 

10.9500 

2.9499 

2.183 

8.0001 

5.333 

1.500 

16 

21.3500 

12.2370 

2.415 

9.1130 

5.232 

1.742 

17 

9.6670 

10.6056 

2.364 

-0.9386 

5.256 

-0.179 

18 

8.3330 

8.9742 

2.349 

-0.6412 

5.262 

-0.122 

19 

6.6670 

7.3428 

2.370 

-0.6768 

5.253 

-0.129 

20 

3.5660 

5.7114 

2.428 

-2.1454 

5.226 

-0.410 

21 

41.9170 

23.4718 

2.428 

18.4452 

5.226 

3.529 

22 

16.8330 

21.8404 

2.370 

-5.0074 

5.253 

-0.953 

23 

10.3330 

20.2090 

2.349 

-9.8760 

5.262 

-1.877 

24 

13.6670 

18.5776 

2.364 

-4.9106 

5.266 

-0.934 

25 

13.5830 

18.9462 

2.415 

-3.3632 

5.232 

-0.643 

26 

20.8500 

18.9278 

2.113 

1.9222 

6.361 

0.359 

27 

16.6670 

17.2963 

1.917 

-0.6293 

5.434 

-0.116 

28 

11.3830 

15.6649 

1.749 

-4.2819 

5.491 

-0.780 

29 

13.7600 

14.0335 

1.618 

-0.2835 

5.531 

-0.061 

30 

11.6830 

12.4021 

1.532 

-0.7191 

5.555 

-0.129 

31 

7.6500 

10.7707 

1.501 

-3.1207 

5.564 

-0.661 

32 

12.1600 

9.1393 

1.526 

3.0107 

5.557 

0.542 

33 

6.7830 

7.5079 

1.607 

-0.7249 

5.534 

-0.131 

34 

5.7000 

5.8765 

1.734 

-0.1765 

5.496 

-0.032 

35 

6.9670 

4.2451 

1.899 

2.7219 

5.441 

0.500 

36 

11.5000 

14.5397 

1.899 

-3.0397 

5.441 

-0.559 

37 

13.7330 

12.9083 

1.734 

0.8247 

5.496 

0.150 

38 

7.6000 

11.2769 

1.607 

-3.6769 

5.534 

-0.664 

39 

5.0670 

9.6455 

1.526 

-4.5785 

5.557 

-0.824 

40 

12.5830 

8.0141 

1.501 

4.5689 

5.564 

0.821 

41 

7.2000 

6.3827 

1.532 

0.8173 

5.555 

0.147 

42 

3.8330 

4.7513 

1.618 

-0.9183 

5.531 

-0.166 

43 

6.6500 

3.1199 

1.749 

3.5301 

5.491 

0.643 

44 

3.1830 

1.4885 

1.917 

1.6945 

5.434 

0.312 

45 

2.9160 

-0.1430 

2.113 

3.0590 

5.361 

0.571 

46 

39.5830 

22.4389 

1.972 

17.1441 

5.415 

3.166 

47 

21.1670 

20.8075 

1.810 

0.3595 

5.471 

0.066 

48 

12.7600 

19.1761 

1.684 

-6.4261 

5.511 

-1.166 

F-6 


49 

12.1670 

17.5447 

1.603 

-5.3777 

5.536 

-0.972 

SO 

14.0830 

15.9133 

1.574 

-1.8303 

6.544 

-0.330 

51 

4.7500 

14.2819 

1.599 

-9.5319 

5.536 

-1.722 

52 

12.2500 

12.6505 

1.677 

-0.4005 

5.513 

-0.073 

53 

7.7500 

11.0191 

1.800 

-3.2691 

5.474 

-0.597 

54 

12.0830 

9.3877 

1.960 

2.6953 

5.419 

0.497 

55 

7.2500 

7.7562 

2.149 

-0.5062 

5.347 

-0.095 

56 

17.0830 

18.0509 

2.003 

-0.9679 

5.404 

-0.179 

67 

15.1670 

16.4194 

1.894 

-1.2524 

5.442 

-0.230 

58 

16.8330 

14.7880 

1.828 

2.0450 

5.465 

0.374 

59 

20.7000 

13.1566 

1.807 

7.5434 

5.472 

1.379 

60 

11.3000 

11.5252 

1.834 

-0.2252 

5.463 

-0.041 
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Cook’s 

Obs  -2-1-0  12  D 


1  1 

1 

0.007 

2  I 

|**** 

0.172 

3  I 

***  | 

0.096 

4  I 

|«* 

0.034 

5  I 

**  1 

0.050 

6  I 

*1 

0.021 

7  I 

♦  | 

0.006 

8  I 

1 

0.000 

9  1 

1 

0.001 

10  I 

**l 

0.015 

11  1 

♦  | 

0.006 

12  I 

1 

0.000 

13  I 

1 

0.000 

14  1 

|* 

0.006 

15  I 

|** 

0.054 

16  I 

!**♦ 

0.092 

17  I 

1 

0.001 

18  I 

1 

0.000 

19  i 

1 

0.000 

20  I 

1 

0.005 

21  I 

1  ****** 

0.384 

22  I 

*  | 

0.026 

23  I 

***  | 

0.100 

24  I 

0.025 

25  I 

♦  | 

0.013 

26  I 

1 

0.003 

27  | 

1 

0.000 

28  1 

*| 

0.009 

29  | 

1 

0.000 

30  I 

1 

0.000 

31  1 

♦  | 

0.003 

32  1 

1  * 

0.003 

33  1 

1 

0.000 

34  1 

1 

0.000 

35  I 

1  * 

0.004 

36  I 

*  1 

0.005 

37  I 

0.000 

38  1 

*1 

0.005 

39  1 

0.007 

40  | 

1* 

0.007 

41  I 

1 

0.000 

42  I 

0.000 

43  I 

1  * 

0.006 

44  I 

1 

0.002 

45  I 

1  * 

0.007 

46  I 

\  ****** 

0.190 

47  I 

1 

0.000 

48  I 

**  | 

0.018 
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49  I  *1  I  0.011 

50  I  I  |  0.001 

51  I  ***l  I  0.035 

52  I  I  I  0.000 

53  I  *1  |  0.006 

54  I  I  I  0.005 

55  I  i  I  0.000 

56  I  I  |  0.001 

57  I  I  |  0.001 

58  I  I  |  0.002 

59  I  I**  |  0.030 

60  I  I  |  0.000 


Sum  of  Residuals 

Sum  of  Squared  Residuals 

Predicted  Resid  SS  (Press) 


0 

1760.0805 

2394.8191 
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EXPERIMENT  1:  6DSS  VERSUS  FACILITATED-MANUAL 
REGRESSIOI  AIALYSIS  USIIG  IIDICATOR  VARIABLES 
IICLUDIMG  FACILITATOR  AS  AI  IVDEPEIDEIT  VARIABLE 


Model:  MODEL 1 
Dependent  Variable:  TIME 

Analysis  of  Variance 


Source 

Sum  of 
DF  Squares 

Mean 

Square 

F  Value 

Prob>F 

Model 

Error 

C  Total 

7  2707.71565 
52  1739.55455 
59  4447.27020 

386.81652 

33.45297 

11.563 

0.0001 

Root  MSE 
Dep  Mean 

C.V. 

5.78385  R-square 
13.21183  Adj  R-sq 
43.77783 

0.6088 

0.5562 

Parameter  Estimates 


Parameter 

Standard 

T  for  HO: 

Variable 

DF 

Estimate 

Error 

Parameters 

Prob  >  iTl 

INTERCEP 

1 

11.668915 

15.9800860? 

0.730 

0.4685 

GROUP 1 

1 

6.490289 

9 . 13553980 

0.710 

0.4806 

GROUP 2 

1 

4.721847 

9.08270204 

0.520 

0.6054 

GR0UP3 

1 

-9.974126 

3.67063696 

-2.717 

0.0089 

DAY 

1 

-5.031379 

1.77820518 

-2.829 

0.0066 

INTERVAL 

1 

-1.698408 

0.29900938 

-5.339 

0.0001 

FACILITY 

1 

10.372573 

4.55731882 

2.276 

0.0270 

FACILITR 

1 

2.873658 

3.66860663 

0.783 

0.4370 

Variance 

Variable 

DF 

Inflation 

INTERCEP 

1 

0.00000000 

GROUP 1 

1 

28.06636971 

GR0UP2 

1 

20.55011136 

GR0UP3 

1 

5.37015590 

DAY 

1 

1.41781737 

INTERVAL 

1 

1.29510022 

FACILITY 

1 

9.31269488 

FACILITR 

1 

32.68819599 

Durbin-Watson  D  i.988 
(For  lumber  of  Obs . )  80 
1st  Order  Autocorrelation  0.004 
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Dep  Var 

Predict 

Obs 

TIME 

Value 

1 

32.0000 

29.8090 

2 

40.6700 

28.2126 

3 

16.5800 

26.6162 

4 

30.2500 

25.0198 

5 

16.3300 

23.4234 

6 

12.6600 

17 . 2787 

7 

13.1000 

15 . 6823 

8 

14.7500 

14.0859 

9 

13.9300 

12.4895 

10 

5.4600 

10.8931 

11 

6.0330 

9.2967 

12 

7.5500 

7.7003 

13 

6.2500 

6.1039 

14 

7.5170 

4.5075 

15 

10.9500 

2.9111 

16 

21.3500 

12.5596 

17 

9.6670 

10.9632 

18 

8.3330 

9.3668 

19 

6 . 6670 

7 . 7704 

20 

3.5660 

6.1740 

21 

41.9170 

23.0092 

22 

16.8330 

21.4128 

23 

10.3330 

19.8164 

24 

13.6670 

18.2200 

25 

13.5830 

16.6236 

26 

20.8500 

19.0919 

27 

16.6670 

17.4955 

28 

11.3830 

15.8991 

29 

13.7600 

14.3027 

30 

11.6830 

12.7063 

31 

7.6500 

11.1099 

32 

12.1500 

9.5135 

33 

6.7830 

7.9171 

34 

5.7000 

6.3207 

35 

6.9670 

4.7243 

36 

11.5000 

14.0605 

37 

13.7330 

12.4641 

38 

7.6000 

10.8677 

39 

5.0670 

9.2713 

40 

12.5830 

7.6749 

41 

7.2000 

6.0785 

42 

3.8330 

4.4821 

43 

6.6500 

2.8857 

44 

3.1830 

1 . 2893 

45 

2.9160 

-0.3071 

46 

39.5830 

21.5671 

47 

21.1670 

19.9707 

48 

12.7600 

18.3743 

F-ll 


Residual 

2.1910 
12.4574 
-10.0362 
5.2302 
-7.0934 
-4.6187 
-2.5823 
0.6641 
1.4405 
-5.4331 
-3.2637 
-0.1503 
0.1461 
3.0095 
8.0389 
8.7904 
-1.2962 
-1.0338 
-1.1034 
-2.6080 
18.9078 
-4.5798 
-9.4834 
-4.5530 
-3.0406 
1.7581 
-0.8285 
-4.5161 
-0.5527 
-1.0233 
-3.4599 
2.6365 
-1.1341 
-0.6207 
2 . 2427 
-2.5605 
1.2689 
-3.2677 
-4.2043 
4.9081 
1.1216 
-0.6491 
3.7643 
1.8937 
3.2231 
18.0159 
1 . 1963 
-5.6243 


49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

Sum  of  Residuals 

Sum  of  Squared  Residuals 

Predicted  Resid  SS  (Press) 


12.1670 

16.7779 

-4.6109 

14.0830 

15.1815 

-1.0985 

4.7500 

13.5851 

-8.8351 

12.2500 

11.9887 

0.2613 

7.7500 

10.3923 

-2.6423 

12.0830 

8.7959 

3.2871 

7.2500 

7.1995 

0.0505 

17.0830 

19.4094 

-2.3264 

15.1670 

17.8130 

-2.6480 

16.8330 

16.2166 

0.6164 

20.7000 

14.6202 

6.0798 

11.3000 

13.0238 

-1.7238 

0 

1739.5546 

2469.0448 
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RESIDUAL 


R 

e 

s 

i 

d 

u 

a 

1 


20  + 


15  + 


10  + 


5  + 


0  + 


*  * 

* 

*  * 


** 


*  * 

*  *  * 

** 


*  * 

* 

*  * 

*  *  * 


-5  + 


-10  + 


*  *  *  *  * 

*  ** 

*  *  ***  * 


-+ - + - + - + - + - + - + - +- 

-5  0  5  10  15  20  25  30 

Predicted  Value  of  TINE  PRED 
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Dep  Var 

Predict 

Std  Err 

Qbs 

TIME 

Value 

Predict 

1 

32.0000 

29.8090 

2.459 

2 

40.6700 

28.2126 

2.416 

3 

16.5800 

26.6162 

2.410 

4 

30.2500 

25.0198 

2.441 

5 

16.3300 

23.4234 

2.507 

6 

12.6600 

17.2787 

2.252 

7 

13.1000 

15.6823 

2.079 

8 

14.7600 

14.0859 

1.939 

9 

13.9300 

12.4895 

1.836 

10 

5.4600 

10.8931 

1.778 

11 

6.0330 

9.2967 

1.770 

12 

7.5500 

7.7003 

1.811 

13 

6.2500 

6.1039 

1.900 

14 

7.6170 

4.6075 

2.029 

15 

10.9500 

2.9111 

2.191 

16 

21.3500 

12.5596 

2.459 

17 

9.6670 

10.9632 

2.416 

18 

8.3330 

9.3668 

2.410 

19 

6.6670 

7.7704 

2.441 

20 

3.6660 

6.1740 

2.607 

21 

41.9170 

23.0092 

2.507 

22 

16.8330 

21.4128 

2.441 

23 

10.3330 

19.8164 

2.410 

24 

13.6670 

18.2200 

2.416 

25 

13.5830 

16.6236 

2.459 

26 

20.8500 

19.0919 

2.131 

27 

16.6670 

17.4965 

1.941 

28 

11.3830 

15.8991 

1.781 

29 

13.7500 

14.3027 

1.659 

30 

11.6830 

12.7063 

1.586 

31 

7.6600 

11.1099 

1.567 

32 

12.1600 

9.5135 

1.605 

33 

6.7830 

7.9171 

1.695 

34 

6.7000 

6.3207 

1.830 

35 

6.9670 

4.7243 

2.001 

36 

11.5000 

14.0605 

2.001 

37 

13.7330 

12.4641 

1.830 

38 

7.6000 

10.8677 

1.695 

39 

5.0670 

9.2713 

1.605 

40 

12.5830 

7.6749 

1.567 

41 

7.2000 

6.0785 

1.586 

42 

3.8330 

4.4821 

1.659 

43 

6.6500 

2.8857 

1.781 

44 

3.1830 

1 . 2893 

1.941 

45 

2.9160 

-0.3071 

2.131 

46 

39.5830 

21.5671 

2.271 

Std  Err  Student 
Residual  Residual  Residual 


2.1910 

5.235 

0.419 

12.4574 

5.255 

2.371 

-10.0362 

5.258 

-1.909 

5.2302 

5.244 

0.997 

-7 . 0934 

5.212 

-1.361 

-4.6187 

5.328 

-0.867 

-2.5823 

5.397 

-0.478 

0.6641 

5.449 

0.122 

1.4405 

5.485 

0.263 

-5.4331 

5.504 

-0.987 

-3 . 2637 

5.506 

-0.693 

-0.1503 

5.493 

-0.027 

0.1461 

5.463 

0.027 

3.0095 

5.416 

0.556 

8.0389 

5.353 

1.502 

8.7904 

5.235 

1.679 

-1.2962 

5.255 

-0.247 

-1.0338 

6.258 

-0.197 

-1.1034 

5.244 

-0.210 

-2.6080 

5.212 

-0.500 

18.9078 

5.212 

3.628 

-4.6798 

5.244 

-0.873 

-9.4834 

5.258 

-1.804 

-4.5530 

5.255 

-0.866 

-3 . 0406 

5.235 

-0.581 

1.7581 

6.377 

0.327 

-0.8285 

5.448 

-0.152 

-4.5161 

5.503 

-0.821 

-0 . 5527 

5.541 

-0.100 

-1.0233 

5.562 

-0.184 

-3.4599 

5.568 

-0.621 

2.6365 

5.557 

0.474 

-1.1341 

5.530 

-0.205 

-0.6207 

5.487 

-0.113 

2 . 2427 

6.427 

0.413 

-2.5605 

5.427 

-0.472 

1 . 2689 

5.487 

0.231 

-3.2677 

5.530 

-0.691 

-4.2043 

5.557 

-0.757 

4.9081 

5.568 

0.882 

1.1215 

5.562 

0.202 

-0.6491 

5.541 

-0.117 

3.7643 

5.503 

0.684 

1.8937 

5.448 

0.348 

3.2231 

5.377 

0.599 

18.0159 

5.320 

3.387 
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47 

21.1670 

19.9707 

2.107 

1.1963 

5.386 

0.222 

48 

12.7500 

18.3743 

1.976 

-5.6243 

5.436 

-1.035 

49 

12.1670 

16.7779 

1.883 

-4.6109 

5.469 

-0.843 

50 

14.0830 

15.1815 

1.835 

-1.0986 

5.485 

-0.200 

51 

4.7500 

13.5851 

1.835 

-8.8351 

5.485 

-1.611 

52 

12.2500 

11.9887 

1.883 

0.2613 

5.469 

0.048 

53 

7.7500 

10.3923 

1.976 

-2.6423 

5.436 

-0.486 

54 

12.0830 

8.7959 

2.107 

3.2871 

5.386 

0.610 

55 

7.2500 

7.1995 

2.271 

0.0505 

5.320 

0.010 

56 

17.0830 

19.4094 

2.655 

-2.3264 

5.139 

-0.453 

67 

15.1670 

17.8130 

2.604 

-2.6460 

5.165 

-0.512 

58 

16.8330 

16.2166 

2.587 

0.6164 

5.173 

0.119 

59 

20.7000 

14.6202 

2.604 

6.0798 

5.165 

1.177 

60 

11.3000 

13.0238 

2.655 

-1.7238 

5.139 

-0.336 
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Cook's 

Obs  -2-1-0  12  D 


1  1 

1 

0.005 

2  I 

|**** 

0.149 

3  I 

***  | 

0.096 

4  I 

0.027 

S  I 

**| 

0.054 

6  I 

*| 

0.017 

7  I 

1 

0.004 

8  I 

1 

0.000 

9  i 

1 

0.001 

10  I 

*  | 

0.013 

11  1 

*| 

0.005 

12  I 

0.000 

13  I 

1 

0.000 

14  ! 

|* 

0.005 

15  I 

|*** 

0.047 

16  I 

|*** 

0.078 

17  I 

1 

0.002 

18  I 

1 

0.001 

19  I 

1 

0.001 

20  I 

*  | 

0.007 

21  I 

| ****** 

0.381 

22  I 

*  | 

0.021 

23  I 

***  | 

0.085 

24  I 

*| 

0.020 

25  I 

*  | 

0.009 

26  I 

1 

0.002 

27  I 

1 

0.000 

28  I 

*  | 

0.009 

29  1 

1 

0.000 

30  I 

1 

0.000 

31  1 

*  | 

0.004 

32  1 

1 

0.002 

33  1 

1 

0.000 

34  1 

1 

0.000 

35  1 

1 

0.003 

36  I 

1 

0.004 

37  1 

1 

0.001 

38  I 

*  | 

0.004 

39  I 

*  | 

0.006 

40  I 

|* 

0.008 

41  I 

1 

0.000 

42  I 

1 

0.000 

43  | 

|* 

0.006 

44  1 

1 

0.002 

45  I 

|* 

0.007 

46  I 

| ****** 

0.261 

47  | 

1 

0.001 

48  I 

**  | 

0.018 
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49  I  *1  I  0.011 

50  |  |  |  0.001 

51  I  ***|  |  0.030 

62  |  |  |  0.000 

S3  |  |  |  0.004 

64  |  1*1  0.007 

66  |  |  1  0.000 

66  |  |  |  0.007 

67  |  *|  |  0.008 

68  |  |  |  0.000 

69  |  |**  |  0.044 

60  |  |  |  0.004 

Sum  of  Residuals  0 

Sum  of  Squared  Residuals  1739.5546 

Predicted  Resid  SS  (Press)  2469.0448 
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EXPERIMEIT  1:  6DSS  VERSUS  FACILITATED-HAHUAL 
TEST  FOR  IORMALITY  OF  RESIDUALS 
RUI  EXP1ARSD  WITHOUT  FACILITATOR 


Univariate  Procedure 


Variable=RESIDUAL 


Moment 8 


I 

60 

Sum  Wgts 

60 

Mean 

0 

Sum 

0 

Std  Dev 

5.461857 

Variance 

29.83189 

Skesness 

1.201903 

Kurtosis 

2.760757 

USS 

1760.081 

CSS 

1760.081 

CV 

. 

Std  Mean 

0.705123 

T:Mean=0 

0 

Prob> I T I 

1.0000 

Hum  "=  0 

SO 

Mum  >  0 

24 

M(Sign) 

-6 

Prob> I M I 

0.1550 

Sgn  Rank 

-132 

Prob> I S I 

0.3354 

W : lormal 

0.910569 

Prob<W 

0.0002 

Quantiles (Del =5) 


100%  Max 

18.4452 

99% 

18.4452 

76%  Q3 

2.2793 

95% 

10 . 964 

60%  Med 

-0.56776 

90% 

6.6006 

25%  Q1 

-3.31616 

10% 

-5.5123 

0%  Min 

-9.876 

6% 

-8.08135 

Range 

q3-qi 

Mode 

28.3212 

5.59545 

-9.876 

1% 

-9.876 

Extremes 


Losest 

Obs 

Highest 

Obs 

-9.876( 

23) 

8. 0001 ( 

15) 

-9. 6436 ( 

3) 

9. 1 13  C 

16) 

-9.5319( 

51) 

12.815( 

2) 

-6.6308( 

5) 

17.1441( 

46) 

-6 . 4261 ( 

48) 

18.4452C 

21) 
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Univariate  Procedure 


Variable=RESIDUAL 


Moments 


1 

60 

Sum  Vgts 

60 

Mean 

0 

Sum 

0 

Std  Dev 

S.4618S7 

Variance 

29.83189 

Skewness 

1.201903 

Xurtosis 

2.760757 

USS 

1760.081 

CSS 

1760.081 

CV 

. 

Std  Mean 

0.705123 

T : Mean=0 

0 

Prob> | T I 

1.0000 

lum  "=  0 

60 

lum  >  0 

24 

M(Sign) 

-6 

Prob> | M I 

0.1550 

Sgn  Rank 

-132 

Prob> I S I 

0.3354 

Quantiles (Del =5) 


100*/.  Max 

18.4452 

997. 

18.4452 

757.  Q3 

2.2793 

957. 

10.964 

507.  Med 

-0.56775 

907. 

6.6006 

257.  Q1 

-3.31615 

107. 

-5.5123 

07.  Min 

-9.876 

57. 

-8.08135 

Range 

Q3-qi 

Mode 

28.3212 

5.59545 

-9.876 

15C 

-9.876 

Extremes 


Lowest 

Obs 

Highest 

Obs 

-9.876( 

23) 

8. 0001  ( 

15) 

-9. 6436 ( 

3) 

9. 1 13  C 

16) 

-9 . 6319( 

51) 

12.815( 

2) 

-6. 6308 ( 

5) 

17.1441C 

46) 

-6. 4261 ( 

48) 

18.4452( 

21) 
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Univariate  Procedure 


Variable-RESIDUAL 


Stem  Leal  # 

18  4  1 

16  1  1 

14 

12  8  1 
10 

8  01  2 

6  5  1 

4  67  2 

2  057790 IS  8 

0  04488279  8 

-0  8309977766543322  16 

-2  74431091  8 

-4  6400963  7 

-6  64  2 

-8  965  3 


Bozplot 

0 

0 

0 

I 


+ - + 

I  +  I 

* - * 

+ - + 

I 

I 


19+ 


formal  Probability  Plot 

* 

* 


*  ++++ 
+++ 


6+ 


-9+  * 

+ — 


*+*++ 

++*+ 

+++** 

******** 

+++**** 

******* 

****+ 

****** 

*  *+++ 

*+*++ 

- + - + - + - + - * - + - + - + - + 

-2  -1  0  +1  +2 
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EXPERIMEIT  1:  GDSS  VERSUS  FACILITATED-MAIUAL 
TEST  FOR  IORMALITY  OF  RESIDUALS 
RU1  EXP1FRSD  WITH  FACILITATOR 


Univariate  Procedure 


Variable-RESIDUAL 


Moment 8 


1 

60 

Sum  Vgts 

60 

Mean 

0 

Sum 

0 

Std  Dev 

5.429916 

Variance 

29.48398 

Skevness 

1.356671 

Kurtosis 

3.374829 

USS 

1739.555 

CSS 

1739.555 

CV 

. 

Std  Mean 

0.700999 

T:Mean=0 

0 

Prob>|T| 

1.0000 

Hum  “=  0 

60 

Hum  >  0 

26 

M(Sign) 

-4 

Prob> | M 1 

0.3663 

Sgn  Rank 

-123 

Prob> | S 1 

0.3696 

V : Hormal 

0.901437 

ProbcH 

0.0001 

quantiles (Def=5) 


1007.  Max 

18.9078 

997. 

18.9078 

757.  Q3 

2.04235 

95% 

10.6239 

507.  Med 

-0.7388 

907. 

5.655 

257.  qi 

-3.15215 

107. 

-5.0259 

07.  Min 

-10.0362 

6% 

-7.96425 

17. 

-10.0362 

Range 

28.944 

Q3-qi 

5.1945 

Mode 

-10.0362 

Extremes 

L0B68t 

Obs 

Highest 

□bs 

-10.0362( 

3) 

8. 0389  ( 

15) 

-9.4834( 

23) 

8 . 7904( 

16) 

-8. 8351 ( 

51) 

12. 4574 ( 

2) 

-7.0934( 

5) 

18.0159( 

46) 

-5. 6243 ( 

48) 

18. 9078 ( 

21) 
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Univariate  Procedure 


Variable=RESIDUAL 


Moment 8 


V 

60 

Sum  Wgts 

60 

Mean 

0 

Sum 

0 

Std  Dev 

6. 4299 16 

Variance 

29.48398 

Skewness 

1.356671 

Kurtosis 

3.374829 

USS 

1739.555 

CSS 

1739.555 

CV 

. 

Std  Mean 

0.700999 

T: Mean-0 

0 

Prob> I T | 

1.0000 

Mum  "=  0 

60 

Rum  >  0 

26 

M(Sign) 

-4 

Prob>|M| 

0.3663 

Sgn  Rank 

-123 

Prob> I S I 

0.3696 

quantiles (Def=5) 

100%  Max 

18.9078 

99% 

18.9078 

75%  Q3 

2.04235 

95% 

10.6239 

50%  Med 

-0.7388 

90% 

5.655 

25%  qi 

-3.15215 

10% 

-5.0259 

0%  Min 

-10.0362 

5% 

-7.96425 

Range 

q3-qi 

Mode 

28.944 

6.1945 

-10.0362 

1% 

-10.0362 

Extremes 


Lowest 

Obs 

Highest 

Obs 

-10.0362( 

3) 

8. 0389 ( 

15) 

-9. 4834 ( 

23) 

8. 7904 ( 

16) 

-8. 8351 ( 

51) 

12. 4574 ( 

2) 

-7. 0934 ( 

5) 

18.0159( 

46) 

-5. 6243 ( 

48) 

18. 9078 ( 

21) 
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Univariate  Procedure 


Variable-RESIDUAL 


Stem  Leal  * 

18  09  2 

16 
14 

12  5  1 

10 

8  08  2 

6  1  1 

4  92  2 

2  2260238  7 

0  11367123489  11 

-0  731110086662  12 

-2  S330666663  10 

-4  64666652  8 

-6  1  1 

-8  58  2 

-10  0  1 

- + - + - + - + 


Bozplot 

* 


0 

I 


+ - + 

I  +  I 

* - * 

+ - + 


19+ 


formal  Probability  Plot 


*  * 


13+ 


*  +++ 


I 

I 

7+ 


1+ 

I 

I 

-5+ 

I 


-11+ 

+ 
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*+*++ 

+*+ 

+++** 

******** 

******* 

***** 

***** 

******* 

*  +++ 

*+*++ 

*++++ 

- + - + - + - + - + - + - + - + - + — 

-2  -1  0  +1  +2 
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EXPERIMENT  1:  GDSS  VERSUS  FACILITATED-MANUAL 
REGRESSION  ANALYSIS  USING  INDICATOR  VARIABLES 
TIME  TRANSFORMED  USING  (TIME) ‘(.5) 

Model:  MODEL 1 
Dependent  Variable:  TIME 

Analysis  of  Variance 
Sun  of  Mean 

Source  DF  Squares  Square  F  Value  Prob>F 

Model  6  45.51752  7.58625  17.871  0.0001 

Error  53  22.49830  0.42450 

C  Total  59  68.01582 

Root  MSE  0.65153  R-square  0.6692 

Dep  Mean  3.47538  Adj  R-sq  0.6318 

C.V.  18.74714 

Parameter  Estimates 


Parameter 

Standard 

T  for  HO: 

Variable 

DF 

Estimate 

Error 

Parameter=0 

Prob  >  ITI 

INTERCEP 

1 

5.004143 

0.24446697 

20.470 

0.0001 

GROUP 1 

1 

-0.070729 

0.26822803 

-0.264 

0.7930 

GR0UP2 

1 

-0.249659 

0.31601228 

-0.790 

0.4330 

GR0UP3 

1 

-1.306354 

0.38751985 

-3.371 

0.0014 

DAY 

1 

-0.566285 

0 . 17766434 

-3.187 

0.0024 

INTERVAL 

1 

-0.226474 

0.03330435 

-6.800 

0.0001 

FACILITY 

1 

0.800725 

0.29800933 

2.687 

0.0096 

Variance 

Variable 

DF 

Inflation 

INTERCEP 

1 

0.00000000 

GROUP 1 

1 

1.90672055 

GR0UP2 

1 

1.96043696 

GR0UP3 

1 

4.71885858 

DAY 

1 

1.11536758 

INTERVAL 

1 

1.26618320 

FACILITY 

1 

3.13817538 

Durbin-Watson  D 

2.027 

(For  Number  ol 

Obs 

.)  60 

1st  Order  Autocorrelation  -0.014 
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Dep  Var 

Predict 

Obs 

TIME 

Value 

Residual 

1 

5.6569 

6.5077 

0.1492 

2 

6.3773 

5.2812 

1.0961 

3 

4.0719 

5.0547 

-0.9829 

4 

5.5000 

4.8282 

0.6718 

5 

4.0410 

4.6018 

-0.5607 

6 

3.5581 

4.1407 

-0.6826 

7 

3.6194 

3.9142 

-0.2948 

8 

3.8406 

3.6877 

0.1529 

9 

3.7323 

3.4612 

0.2711 

10 

2.3367 

3.2348 

-0.8981 

11 

2.4562 

3.0083 

-0.5521 

12 

2.7477 

2.7818 

-0.0341 

13 

2.5000 

2.5553 

-0.0553 

14 

2.7417 

2.3289 

0.4129 

15 

3.3091 

2.1024 

1.2067 

16 

4.6206 

3 . 3956 

1.2250 

17 

3.1092 

3.1692 

-0.0600 

18 

2.8867 

2.9427 

-0.0560 

19 

2.5821 

2.7162 

-0.1342 

20 

1.8884 

2.4897 

-0.6014 

21 

6.4743 

4.7625 

1.7119 

22 

4.1028 

4.5360 

-0.4332 

23 

3.2145 

4.3095 

-1.0950 

24 

3.6969 

4.0830 

-0.3861 

25 

3.6855 

3.8566 

-0.1710 

26 

4.5662 

4.2720 

0.2941 

27 

4.0825 

4.0456 

0.0370 

28 

3.3739 

3.8191 

-0.4452 

29 

3.7081 

3.5926 

0.1155 

30 

3.4180 

3.3661 

0.0519 

31 

2.7659 

3.1397 

-0.3738 

32 

3.4857 

2.9132 

0.5725 

33 

2.6044 

2.6867 

-0.0823 

34 

2.3875 

2.4602 

-0.0728 

35 

2 . 6395 

2.2338 

0.4057 

36 

3.3912 

3.7058 

-0.3146 

37 

3.7058 

3.4793 

0.2265 

38 

2.7568 

3.2528 

-0.4960 

39 

2.2510 

3.0263 

-0.7753 

40 

3.5473 

2 . 7999 

0.7474 

41 

2.6833 

2.5734 

0.1099 

42 

1 . 9578 

2 . 3469 

-0.3891 

43 

2.5788 

2.1204 

0.4583 

44 

1.7841 

1.8940 

-0.1099 

45 

1.7076 

1.6675 

0.0401 

46 

6.2915 

4.7777 

1.5138 

47 

4.6008 

4.5512 

0.0496 

48 

3.6707 

4.3247 

-0.7540 
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49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

Sum  of  Residuals 

Sum  of  Squared  Residuals 

Predicted  Resid  SS  (Press) 


3.4881 

4.0982 

-0.6101 

3.7527 

3.8718 

-0.1190 

2.1794 

3.6453 

-1.4658 

3.5000 

3.4188 

0.0812 

2.7839 

3.1923 

-0.4085 

3.4761 

2.9659 

0.5102 

2.6926 

2.7394 

-0.0468 

4.1332 

4.2114 

-0.0782 

3.8945 

3.9849 

-0.0904 

4.1028 

3.7584 

0.3444 

4.5497 

3.5320 

1.0178 

3.3615 

3 . 3055 

0.0561 

0 

22.4983 

29.9063 


R 

e 

s 

i 

d 

u 

a 

1 


RESIDUAL 


2.0 


1.5 


1.0 


0.5 


0.0 


-0.5 


-1.0 


-1.5 


1.5 


2.0  2.5  3.0  3.5  4.0 

Predicted  Value  of  TINE 


4.6 

PRED 


5.0 


5.5 
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Dep  Var 

Predict 

Std  Err 

Obs 

TIME 

Value 

Predict 

i 

5.6569 

5.5077 

0.273 

2 

6.3773 

5.2812 

0.267 

3 

4.0719 

5.0647 

0.266 

4 

5.5000 

4.8282 

0.268 

5 

4.0410 

4.6018 

0.274 

6 

3.5581 

4.1407 

0.248 

7 

3.6194 

3.9142 

0.230 

8 

3.8406 

3.6877 

0.215 

9 

3.7323 

3.4612 

0.204 

10 

2.3367 

3.2348 

0.198 

11 

2.4562 

3.0083 

0.198 

12 

2.7477 

2.7818 

0.203 

13 

2.5000 

2.5553 

0.213 

14 

2.7417 

2.3289 

0.228 

15 

3.3091 

2.1024 

0.247 

16 

4.6206 

3.3956 

0.273 

17 

3.1092 

3.1692 

0.267 

18 

2.8867 

2.9427 

0.266 

19 

2.5821 

2.7162 

0.268 

20 

1.8884 

2.4897 

0.274 

21 

6.4743 

4.7625 

0.274 

22 

4.1028 

4.5360 

0.268 

23 

3.2145 

4.3095 

0.266 

24 

3.6969 

4.0830 

0.267 

25 

3.6855 

3.8566 

0.273 

26 

4.5662 

4.2720 

0.239 

27 

4.0825 

4.0456 

0.217 

28 

3.3739 

3.8191 

0.198 

29 

3.7081 

3.5926 

0.183 

30 

3.4180 

3.3661 

0.173 

31 

2.7659 

3.1397 

0.170 

32 

3.4857 

2.9132 

0.173 

33 

2.6044 

2.6867 

0.182 

34 

2.3875 

2.4602 

0.196 

35 

2.6395 

2.2338 

0.215 

36 

3.3912 

3.7058 

0.215 

37 

3.7058 

3.4793 

0.196 

38 

2.7568 

3.2528 

0.182 

39 

2.2510 

3.0263 

0.173 

40 

3.5473 

2.7999 

0.170 

41 

2.6833 

2.5734 

0.173 

42 

1 . 9578 

2.3469 

0.183 

43 

2.6788 

2 . 1204 

0.198 

44 

1.7841 

1.8940 

0.217 

45 

1.7076 

1.6675 

0.239 

46 

6.2915 

4.7777 

0.223 

47 

4.6008 

4.6512 

0.205 

48 

3.5707 

4.3247 

0.190 

Std  Err  Student 
Residual  Residual  Residual 


0 . 1492 

0.592 

0.262 

1.0961 

0.594 

1.845 

-0.9829 

0.595 

-1.652 

0.6718 

0.594 

1.131 

-0.5607 

0.591 

-0.949 

-0.5826 

0.602 

-0.967 

-0.2948 

0.610 

-0.484 

0.1529 

0.615 

0.248 

0.2711 

0.619 

0.438 

-0.8981 

0.621 

-1.447 

-0.5521 

0.621 

-0.889 

-0.0341 

0.619 

-0.055 

-0.0553 

0.616 

-0.090 

0.4129 

0.610 

0.877 

1 . 2067 

0.603 

2.001 

1 . 2250 

0.592 

2.071 

-0.0600 

0.594 

-0.101 

-0.0560 

0.595 

-0.094 

-0.1342 

0.594 

-0.226 

-0.6014 

0.591 

-1.018 

1.7119 

0.591 

2.897 

-0.4332 

0.594 

-0.729 

-1.0950 

0.595 

-1.840 

-0.3861 

0.594 

-0.650 

-0.1710 

0.592 

-0.289 

0.2941 

0.606 

0.485 

0.0370 

0.614 

0.060 

-0.4452 

0.621 

-0.717 

0.1155 

0.626 

0.185 

0.0619 

0.628 

0.083 

-0.3738 

0.629 

-0.594 

0 . 5725 

0.628 

0.911 

-0.0823 

0.626 

-0.132 

-0.0728 

0.621 

-0.117 

0.4057 

0.615 

0.660 

-0.3146 

0.615 

-0.511 

0.2265 

0.621 

0.365 

-0.4960 

0.626 

-0.793 

-0.7753 

0.628 

-1.234 

0.7474 

0.629 

1.188 

0.1099 

0.628 

0.175 

-0.3891 

0.625 

-0.622 

0.4583 

0.621 

0.738 

-0.1099 

0.614 

-0.179 

0.0401 

0.606 

0.066 

1.5138 

0.612 

2.473 

0.0496 

0.619 

0.080 

-0 . 7540 

0.623 

-1.210 
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49 

3.4881 

4.0982 

0.181 

-0.6101 

0.626 

-0.976 

EO 

3.7627 

3.8718 

0.178 

-0.1190 

0.627 

-0.190 

SI 

2.1794 

3.6453 

0.181 

-1.4658 

0.626 

-2.342 

S2 

3.5000 

3.4188 

0.190 

0.0812 

0.623 

0.130 

53 

2.7839 

3.1923 

0.204 

-0.4085 

0.619 

-0.660 

54 

3.4761 

2.9659 

0.222 

0.5102 

0.613 

0.833 

SB 

2.6926 

2.7394 

0.243 

-0.0468 

0.605 

-0.077 

56 

4.1332 

4.2114 

0.226 

-0.0782 

0.611 

-0.128 

67 

3.8945 

3.9849 

0.214 

-0.0904 

0.615 

-0.147 

58 

4.1028 

3.7584 

0.207 

0.3444 

0.618 

0.657 

59 

4.5497 

3.5320 

0.204 

1.0178 

0.619 

1.645 

60 

3.3615 

3.3055 

0.207 

0.0561 

0.618 

0.091 
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Cook’s 

Obs  -2-1-0  12  D 


1  1 

1 

0.002 

2  I 

0.098 

3  I 

***| 

0.078 

4  I 

1  ** 

0.037 

6  I 

*  | 

0.028 

6  I 

0.023 

7  I 

1 

0.005 

8  I 

1 

0.001 

9  I 

1 

0.003 

10  I 

**  1 

0.030 

11  1 

♦  | 

0.011 

12  I 

1 

0.000 

13  I 

1 

0.000 

14  I 

|* 

0.009 

15  1 

|**** 

0.096 

16  I 

|  **** 

0.131 

17  I 

1 

0.000 

18  I 

1 

0.000 

19  I 

1 

0.001 

20  I 

**  | 

0.032 

21  I 

| ***** 

0.259 

22  I 

*| 

0.016 

23  I 

***  | 

0.096 

24  1 

*| 

0.012 

25  I 

1 

0.003 

26  | 

1 

0.005 

27  I 

1 

0.000 

28  I 

*  | 

0.007 

29  | 

1 

0.000 

30  1 

1 

0.000 

31  1 

*  | 

0.004 

32  1 

|  * 

0.009 

33  1 

1 

0.000 

34  1 

1 

0.000 

36  1 

1  * 

0.008 

36  1 

*  1 

0.005 

37  1 

1 

0.002 

38  1 

*  | 

0.008 

39  | 

**| 

0.016 

40  I 

|  ** 

0.015 

41  I 

1 

0.000 

42  I 

*  | 

0.005 

43  I 

1* 

0.008 

44  I 

1 

0.001 

45  I 

1 

0.000 

46  I 

|  **** 

0.116 

47  | 

1 

0.000 

48  I 

**  | 

0.020 
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49  | 

*| 

0.011 

50  | 

1 

0.000 

51  I 

****| 

0.065 

52  | 

1 

0.000 

53  | 

*| 

0.007 

54  | 

|* 

0.013 

55  | 

1 

0.000 

56  | 

1 

0.000 

57  | 

1 

0.000 

58  | 

|* 

0.005 

59  | 

|*** 

0.042 

60  | 

1 

0.000 

Sum  ol  Residuals  0 
Sum  ol  Squared  Residuals  22.4983 
Predicted  Resid  SS  (Press)  29.9063 
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EXPERIMEIT  1:  GDSS  VERSUS  FACILITATED-MAIUAL 
REGRESSIOI  ANALYSIS  USIIG  IIDICATOR  VARIABLES 
IICLUDIIG  FACILITATOR  AID  TIME  TRAISFORMED  USIIG  (TIME) ‘(.5) 


Model:  MODEL 1 
Dependent  Variable:  TIME 


Analysis  of  Variance 


Source  DF 

Model  7 
Error  52 
C  Total  69 


Sub  of  Mean 
Squares  Square 

46.14568  6.59224 
21.87C14  0.42058 
68.01582 


F  Value  Prob>F 

15.674  0.0001 


Root  MSE  0.64852  R-square  0.6785 
Dep  Mean  3.47538  Adj  R-sq  0.6352 
C.V.  18.66046 


Parameter  Estimates 


Variable 

DF 

Parameter 

Estimate 

Standard 

Error 

T  for  HO: 

Parameters 

Prob  >  ITI 

IITERCEP 

1 

2.834676 

1.79178494 

1.582 

0.1197 

GROUP 1 

1 

1 . 137846 

1.02433257 

1.111 

0.2718 

GR0UP2 

1 

0.934092 

1.01840808 

0.917 

0.3633 

GR0UP3 

1 

-1.130918 

0.41157426 

-2.748 

0.0082 

DAY 

1 

-0.678827 

0.19938324 

-3.405 

0.0013 

IHTERVAL 

1 

-0.220352 

0.03352676 

-6.572 

0.0001 

FACILITY 

1 

1.309223 

0.51099445 

2.562 

0.0133 

FACILITR 

1 

0.502710 

0.41134660 

1.222 

0.2272 

Variance 


Variable 

DF 

Inflation 

IITERCEP 

1 

0.00000000 

GROUP 1 

1 

28.06636971 

GR0UP2 

1 

20.55011136 

GR0UP3 

1 

6.37015590 

DAY 

1 

1.41781737 

IITERVAL 

1 

1.29510022 

FACILITY 

1 

9.31269488 

FACILITR 

1 

32.68819599 

Durbin-Watson  D 

2.134 

(For  lumber  of 

Obs . 

.)  60 

1st  Order  Autocorrelation  -0.068 

F-32 


F-33 


49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

Sum  of  Residuals 

Sum  of  Squared  Residuals 

Predicted  Resid  SS  (Press) 


3.4881 

3.9641 

-0.4760 

3.7527 

3.7438 

0.00898 

2.1794 

3.5234 

-1.3440 

3.5000 

3.3031 

0.1969 

2.7839 

3.0827 

-0.2988 

3.4761 

2.8623 

0.6137 

2.6926 

2.6420 

0.0506 

4.1332 

4.4490 

-0.3159 

3.8945 

4.2287 

-0.3342 

4.1028 

4.0083 

0.0945 

4.5497 

3.7880 

0.7617 

3.3615 

3.5676 

-0.2061 

0 

21.8701 

30.3491 
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RESIDUAL 


-+ - +- 


-+ - +- - + - +- 


2.0  + 


1.5  + 


1.0  + 


R 

e 

s 

i 

d 

u 

a 

1 


0.5  + 


* 

* 


0.0  + 


* 

*  * 


*  * 
* 


* 

* 

*  * 


*  * 


*  * 


*  *  * 


-0.5  + 


**  * 


-1.0  + 


-1.5  + 


-+ - + - + - + - + - + - + - + - + - + — 

1.5  2.0  2.5  3.0  3.5  4.0  4.5  5.0  5.5  6.0 

Predicted  Value  oi  TIME  PRED 
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Dep  Var 

Predict 

Std  Err 

Obs 

TIME 

Value 

Predict 

1 

5.6569 

5.5641 

0.276 

2 

6.3773 

5.3438 

0.271 

3 

4.0719 

5.1234 

0.270 

4 

5.5000 

4.9030 

0.274 

S 

4.0410 

4.6827 

0.281 

6 

3.5581 

4.0788 

0.252 

7 

3.6194 

3.8584 

0.233 

8 

3.8406 

3.6381 

0.217 

9 

3.7323 

3.4177 

0.206 

10 

2.3367 

3.1974 

0.199 

11 

2.4562 

2.9770 

0.198 

12 

2.7477 

2.7567 

0.203 

13 

2.5000 

2.5363 

0.213 

14 

2.7417 

2.3159 

0.227 

15 

3.3091 

2.0956 

0.246 

16 

4.6206 

3.4521 

0.276 

17 

3.1092 

3.2317 

0.271 

18 

2.8867 

3.0114 

0.270 

19 

2.5821 

2.7910 

0.274 

20 

1.8884 

2.5707 

0.281 

21 

6.4743 

4.6815 

0.281 

22 

4.1028 

4.4612 

0.274 

23 

3.2145 

4.2408 

0.270 

24 

3.6969 

4.0205 

0.271 

25 

3.6855 

3.8001 

0.276 

26 

4.5662 

4.3008 

0.239 

27 

4.0825 

4.0804 

0.218 

28 

3.3739 

3.8601 

0.200 

29 

3.7081 

3.6397 

0.186 

30 

3.4180 

3.4194 

0.178 

31 

2.7659 

3.1990 

0.176 

32 

3.4857 

2.9786 

0.180 

33 

2.6044 

2.7583 

0.190 

34 

2.3875 

2.5379 

0.205 

35 

2.6395 

2.3176 

0.224 

36 

3.3912 

3.6219 

0.224 

37 

3.7058 

3.4016 

0.205 

38 

2.7568 

3.1812 

0.190 

39 

2.2510 

2.9609 

0.180 

40 

3.5473 

2.7405 

0.176 

41 

2.6833 

2.5202 

0.178 

42 

1.9578 

2.2998 

0.186 

43 

2.5788 

2.0795 

0.200 

44 

1.7841 

1.8591 

0.218 

45 

1.7076 

1.6388 

0.239 

46 

6.2915 

4.6252 

0.255 

47 

4.6008 

4.4048 

0.236 

48 

3.5707 

4.1845 

0.222 

Std  Err  Student 
Residual  Residual  Residual 


0.0928 

0.587 

0.158 

1.0336 

0.589 

1.764 

-1.0515 

0.590 

-1.784 

0.5970 

0.588 

1.015 

-0.6417 

0.584 

-1.098 

-0.5207 

0.697 

-0.872 

-0.2390 

0.605 

-0.395 

0.2025 

0.611 

0.331 

0.3146 

0.615 

0.612 

-0.8607 

0.617 

-1.395 

-0.5208 

0.617 

-0.843 

-0.00893 

0.616 

-0.014 

-0.0363 

0.613 

-0.059 

0.4258 

0.607 

0.701 

1.2135 

0.600 

2.022 

1.1685 

0.587 

1.991 

-0.1225 

0.589 

-0.208 

-0.1247 

0.590 

-0.211 

-0.2090 

0.588 

-0.355 

-0.6823 

0.584 

-1.167 

1.7928 

0.584 

3.068 

-0.3584 

0.588 

-0.610 

-1.0263 

0.590 

-1.741 

-0.3236 

0.589 

-0.549 

-0.1146 

0.587 

-0.195 

0.2654 

0.603 

0.440 

0.00212 

0.611 

0.003 

-0.4862 

0.617 

-0.788 

0.0684 

0.621 

0.110 

-0.00131 

0.624 

-0.002 

-0.4331 

0.624 

-0.694 

0.5070 

0.623 

0.814 

-0.1539 

0.620 

-0.248 

-0.1505 

0.615 

-0.245 

0.3219 

0.608 

0.529 

-0.2308 

0.608 

-0.379 

0.3042 

0.615 

0.495 

-0.4244 

0.620 

-0.684 

-0.7099 

0.623 

-1.139 

0.8067 

0.624 

1.292 

0.1631 

0.624 

0.262 

-0.3420 

0.621 

-0.551 

0 . 4993 

0.617 

0.809 

-0.0750 

0.611 

-0.123 

0.0689 

0.603 

0.114 

1 . 6663 

0.596 

2.794 

0.1959 

0.604 

0.324 

-0.6137 

0.610 

-1.007 
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49 

3.4881 

3.9641 

0.211 

-0.4760 

0.813 

-0.776 

SO 

3.7527 

3.7438 

0.206 

0.00898 

0.615 

0.016 

51 

2.1794 

3.5234 

0.206 

-1.3440 

0.615 

-2.185 

52 

3.5000 

3.3031 

0.211 

0.1969 

0.613 

0.321 

53 

2.7839 

3.0827 

0.222 

-0.2988 

0.610 

-0.490 

54 

3.4761 

2.8623 

0.236 

0.6137 

0.604 

1.016 

55 

2.6926 

2.6420 

0.255 

0.0506 

0.596 

0.085 

56 

4.1332 

4.4490 

0.298 

-0.3159 

0.676 

-0.548 

57 

3.8945 

4.2287 

0.292 

-0.3342 

0.579 

-0.577 

58 

4.1028 

4.0083 

0.290 

0.0945 

0.580 

0.163 

59 

4.5497 

3.7880 

0.292 

0.7617 

0.579 

1.315 

60 

3.3615 

3.5676 

0.298 

-0.2061 

0.576 

-0.358 
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Cook’s 

Obs  -2-1-0  12  D 


1  1 

1  1 

0.001 

2  I 

1***  1 

0.081 

3  I 

***|  | 

0.084 

4  I 

1**  1 

0.028 

5  I 

**  1  1 

0.035 

6  I 

*  1  ! 

0.017 

7  I 

1  1 

0.003 

8  I 

1  1 

0.002 

9  I 

1  *  1 

0.004 

10  1 

**  1  1 

0.025 

11  1 

*  1  1 

0.009 

12  I 

1  1 

0.000 

13  I 

1  1 

0.000 

14  I 

1  *  1 

0.009 

15  | 

| ****  | 

0.086 

16  I 

1  ***  1 

0.109 

17  I 

1  1 

0.001 

18  I 

1  1 

0.001 

19  I 

1  1 

0.003 

20  | 

**  1  1 

0.039 

21  I 

j  ******  | 

0.272 

22  I 

*  1  1 

0.010 

23  I 

♦  **  |  | 

0.080 

24  | 

*  |  | 

0.008 

25  | 

1  1 

0.001 

26  1 

1  1 

0.004 

27  I 

1  1 

0.000 

28  I 

*  1  1 

0.008 

29  I 

1  1 

0.000 

30  I 

1  1 

0.000 

31  I 

*  1  1 

0.005 

32  1 

1*  1 

0.007 

33  1 

1  1 

0.001 

34  1 

1  1 

0.001 

35  I 

1  *  1 

0.005 

36  I 

1  1 

0.002 

37  | 

1  1 

0.003 

38  I 

*  1  1 

0.006 

39  | 

**  1  1 

0.014 

40  | 

1  **  1 

0.017 

41  I 

1  i 

0.001 

42  I 

*  1  1 

0.003 

43  | 

1  *  1 

0.009 

44  I 

1  1 

0.000 

45  I 

1  1 

0.000 

46  | 

1#****  | 

0.178 

47  I 

I  1 

0.002 

48  1 

**  1  1 

0.017 
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49  1 

*1 

1  0.009 

SO  1 

1 

1  0.000 

51  1 

***«) 

1  0.067 

52  1 

1 

1  0.002 

53  1 

1 

1  0.004 

54  1 

I** 

1  0.020 

55  1 

1 

1  0.000 

56  I 

*| 

1  0.010 

57  I 

*1 

1  0.011 

58  1 

1 

i  0.001 

59  1 

|** 

1  0.055 

60  1 

1 

1  0.004 

Sum  ol  Residuals  0 
Sum  ol  Squared  Residuals  21.8701 
Predicted  Resid  SS  (Press)  30.3491 
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EXPERIMENT  1:  GDSS  VERSUS  FACILITATED-MANUAL 
TEST  FOR  IORMALITY  OF  RESIDUALS 
RUB  EXP1VRSD  TRANSFORMED  WITHOUT  FACILITATOR 


Univariate  Procedure 

Variable=RESIDUAL 


Moments 


H 

60 

Sum  Wgts 

60 

Mean 

5E-6 

Sum 

0.0003 

Std  Dev 

0.617519 

Variance 

0.38133 

Skewness 

0.537531 

Kurtosis 

0.802214 

USS 

22.49846 

CSS 

22.49846 

CV 

12350381 

Std  Mean 

0.079721 

T:Mean=0 

0.000063 

Prob>lT| 

1.0000 

Hum  *=  0 

60 

Hum  >  0 

27 

M(Sign) 

-3 

Prob> I M 1 

0.5190 

Sgn  Rank 

-71.5 

Prob> I S I 

0.6028 

W : Normal 

0.966709 

Prob<W 

0.2191 

quantiles (Def=5) 

100%  Max 

1.7119 

99% 

1.7119 

75*/.  Q3 

0.2826 

95 % 

1.21585 

50/C  Med 

-0.05665 

90% 

0.8826 

25 y.  qi 

-0.3988 

10% 

-0.68205 

0%  Min 

-1.4658 

5% 

-0.9405 

Range 

Q3-qi 

Mode 

3.1777 

0.6814 

-1.4658 

1% 

-1.4668 

Extremes 


Lowest 

Obs 

Highest 

Obs 

-1.4658( 

51) 

1.0961( 

2) 

-1.095( 

23) 

1.2067( 

16) 

-0.9829( 

3) 

1 . 225 ( 

16) 

-0.898K 

10) 

1.5138C 

46) 

-0.7753( 

39) 

1.7119( 

21) 
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Univariate  Procedure 


Variable=RESIDUAL 


Moments 


1 

60 

Sum  Vgts 

60 

Mean 

5E-6 

Sum 

0.0003 

Std  Dev 

0.617519 

Variance 

0.38133 

Skewness 

0.537531 

Kurtasis 

0.802214 

USS 

22.49846 

CSS 

22.49846 

CV 

12350381 

Std  Mean 

0.079721 

T:Mean=0 

0.000063 

Prob>|Tl 

1.0000 

lum  "=  0 

60 

lum  >  0 

27 

M(Sign) 

-3 

Prob> | M 1 

0.5190 

Sgn  Rank 

-71.5 

Prob> | S 1 

0.6028 

Quantiles (Def=5) 

100'/.  Max 

1.7119 

99% 

1.7119 

75'/.  Q3 

0.2826 

95% 

1.21585 

50%  Med 

-0.05565 

90% 

0.8826 

25%  qi 

-0.3988 

10% 

-0.68205 

0%  Min 

-1.4658 

5% 

-0.9405 

Range 

Q3-Q1 

Mode 

3.1777 

0.6814 

-1.4668 

1% 

-1.4658 

Extremes 


Lowest 

□bs 

Highest 

Obs 

-1.46S8( 

51) 

1.0961C 

2) 

-1.095( 

23) 

1 . 2067 ( 

15) 

-0.9829( 

3) 

1 . 225 ( 

16) 

-0.8981( 

10) 

1.5138C 

46) 

-0.7753( 

39) 

1.7119C 

21) 
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Univariate  Procedure 


Variable=RESIDUAL 


Stem  Leaf 

* 

Boxplot 

16  1 

1 

0 

14  1 

1 

0 

12  12 

2 

1 

10  20 

2 

1 

8 

1 

6  75 

2 

1 

4  11617 

5 

1 

2  3794 

4 

+ - + 

0  4455681255 

10 

1  +  1 

-0  7321988766653 

13 

* - * 

-2  99719 

5 

1  1 

-4  8650531 

7 

+ - + 

-6  8510 

4 

1 

-8  80 

2 

1 

-10  0 

1 

1 

-12 

-14  7 

1 

0 

- + - + - + - + 

Multiply  Stem. Leaf  by  10**-1 


1.7+ 


0.1+ 


Hormal  Probability  Plot 

* 

*  +++ 
**  +++ 

**  +++ 

++++ 

+** 

***** 

***** 

****** 

****** 

*** 

***** 

***** 

***** 

**+ 

*** 


-1.5+++* 

+ - + - + - + - + - + - + - + - + - + - + 

-2  -1  0  +1  +2 
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EXPERIMENT  1:  GDSS  VERSUS  FACILITATED-MAVUAL 
TEST  FOR  NORMALITY  OF  RESIDUALS 
RUN  EXP1WFRD  TRANSFORMED  WITH  FACILITATOR 


Univariate  Procedure 


Variable=RESIDUAL 


Moments 


N 

60 

Sum  Vgts 

60 

Mean 

-2.33E-6 

Sum 

-0.00014 

Std  Dev 

0.608834 

Variance 

0.370679 

Skewness 

0.7260S8 

Kurtosis 

1 . 180389 

USS 

21.87006 

CSS 

21.87006 

CV 

-2 . 609E7 

Std  Mean 

0.0786 

T:Mean=0 

-0.00003 

Prob> I T I 

1.0000 

Rum  '=  0 

60 

Rum  >  0 

27 

M(Sign) 

-3 

Prob> I M 1 

0.5190 

Sgn  Rank 

-89.5 

Prob> I S I 

0.5145 

W: Normal 

0.960295 

Prob<M 

0.1086 

Quantiles(Def=5) 


100*/.  Max 

1.7928 

99% 

1.7928 

75'/.  Q3 

0.2848 

95% 

1.191 

50*/.  Med 

-0.05665 

90% 

0.7842 

25%  Q1 

-0.3502 

10% 

-0.662 

0%  Min 

-1.344 

5% 

-0.9435 

Range 

Q3-Q1 

Mode 

3.1368 

0.635 

-1.344 

1% 

-1.344 

Extremes 


Lowest 

Obs 

Highest 

Obs 

-1 . 344 ( 

51) 

1 . 0336 ( 

2) 

-1.0515( 

3) 

1 . 1685 ( 

16) 

-1.0263( 

23) 

1.2135( 

15) 

-0 . 8607 ( 

10) 

1 . 6663( 

46) 

-0.7099( 

39) 

1 . 7928( 

21) 
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Univariate  Procedure 


Variable=RESIDUAL 


Moments 


I 

60 

Sum  Vgts 

60 

Mean 

-2.33E-6 

Sum 

-0.00014 

Std  Dev 

0.608834 

Variance 

0.370679 

Skewness 

0.726058 

Kurtosis 

1.180389 

USS 

21.87006 

CSS 

21.87006 

CV 

-2.609E7 

Std  Mean 

0.0786 

T : Mean=0 

-0.00003 

Prob> I T I 

1.0000 

■um  "=  0 

60 

Rum  >  0 

27 

M(Sign) 

-3 

Prob> I M I 

0.5190 

Sgn  Rank 

-89.5 

Prob> I S I 

0.5145 

Quantiles (Def=5) 


100'/.  Max 

1.7928 

997. 

1.7928 

757.  q3 

0.2848 

957. 

1.191 

507.  Med 

-0.05565 

907. 

0.7842 

257.  Q1 

-0.3502 

107. 

-0.662 

07.  Min 

-1.344 

57. 

-0.9435 

Range 

Q3-qi 

Mode 

3.1368 

0.635 

-1.344 

17. 

-1.344 

Extremes 


Lowest 

Obs 

Highest 

□bs 

-1.344( 

51) 

1 . 0336( 

2) 

-1.0515( 

3) 

1.1685( 

16) 

-1.0263( 

23) 

1.2135C 

15) 

-0 . 8607 ( 

10) 

1 . 6663( 

46) 

-0.7099( 

39) 

1.792B( 

21) 
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Univariate  Procedure 


Variable=RESIDUAL 


Stem  Leal 

* 

Boxplot 

16  79 

2 

0 

14 

12  1 

1 

1 

10  37 

2 

1 

8  1 

1 

1 

6  016 

3 

1 

4  301 

3 

1 

2  0007012 

7 

+ - + 

0  01577996 

8 

1  1 

-0  552218410 

9 

* — + — * 

-2  6432204311 

10 

+ - + 

-4  229832 

6 

1 

-6  1841 

4 

1 

-8  6 

1 

1 

-10  53 

2 

1 

-12  4 

1 

0 

- + - + - + - + 

Multiply  Stem. Leal  by  10++-1 


1.7+ 

I 

I 

1.1+ 


0.5+ 


-0.1+ 


-0.7+ 


lormal  Probability  Plot 

*  * 

+++ 

*  +++ 

*  *+++ 

*+++ 

+**+ 

++*** 

++*** 

***** 

***** 

***** 

***** 

***** 

*** 

*** 


-1.3+  *+++ 

+ - + - + - + - + - + - + - + - + - + - + 

-2  -1  0  +1  +2 
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EXPERIMENT  2:  FACILITATED-MANUAL  VERSUS  KAIUAL 
REGRESSION  ANALYSIS  USING  INDICATOR  VARIABLES 
WITH  RESIDUAL  PLOT 


Model:  MODEL1 
Dependent  Variable:  TIME 


Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Prob>F 

Model 

6 

2125.72031 

364.28672 

8.675 

0.0001 

Error 

68 

2777.01490 

40.83846 

C  Total 

74 

4902.73521 

Root  MSE  6.390S0  R- square  0.4336 
Dep  Mean  11.17332  Adj  R-sq  0.3836 
C.V.  67.19426 

Parameter  Estimates 


Parameter 

Standard 

T  for  HO: 

Variable 

DF 

Estimate 

Error 

Parameters 

Prob  >  |T| 

INTERCEP 

1 

16.175133 

2.19422853 

7.372 

0.0001 

GR0UP4 

1 

2.571193 

2.99433408 

0.859 

0.3935 

GR0UP5 

1 

1.778550 

2.25938195 

0.787 

0.4339 

GR0UP6 

1 

3.657350 

2.25938195 

1.619 

0.1101 

DAY 

1 

-1.433353 

1.49716704 

-0.957 

0.3418 

INTERVAL 

1 

-1.511237 

0.25990831 

-5.815 

0.0001 

FACILITY 

1 

3.778300 

2.02085265 

1.870 

0.0658 

Variance 

Variable 

DF 

Inflation 

INTERCEP 

1 

0.00000000 

GR0UP4 

1 

2.63458647 

GR0UP6 

1 

1.83333333 

GR0UP6 

1 

1.83333333 

DAY 

1 

1.02456140 

INTERVAL 

1 

1.02005013 

FACILITY 

1 

1.86666667 

Durbin-Watson 

D 

1.961 

(For  Humber  of  Obs . )  76 

1st  Order  Autocorrelation  -0.046 
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Dap  Var 

Pradict 

Oba 

TIME 

Value 

Residual 

1 

39.5830 

21.0134 

18.5696 

2 

21.1670 

19.5022 

1.6648 

3 

12.7500 

17.9909 

-5.2409 

4 

12.1670 

16.4797 

-4.3127 

5 

14.0830 

14.9684 

-0.8854 

6 

4.7600 

13.4572 

-8.7072 

7 

12.2500 

11.9460 

0.3040 

8 

7.7600 

10.4347 

-2.6847 

9 

12.0830 

8.9235 

3.1595 

10 

7.2500 

7.4123 

-0.1623 

11 

17.0830 

19.5800 

-2.4970 

12 

15.1670 

18.0688 

-2.9018 

13 

16.8330 

16.5576 

0.2754 

14 

20.7000 

15.0463 

5.6537 

15 

11.3000 

13.5351 

-2.2351 

16 

40.4160 

20.2207 

20.1953 

17 

24.8330 

18.7095 

6.1235 

18 

9.6670 

17.1983 

-7.5313 

19 

11.5000 

15.6870 

-4.1870 

20 

8.9500 

14.1758 

-5.2268 

21 

8.1600 

12.6646 

-4.6046 

22 

8.5000 

11.1533 

-2.6533 

23 

6.4160 

9.6421 

-3.2261 

24 

8.4160 

8.1308 

0.2852 

25 

5.6000 

6.6196 

-1.1196 

26 

8.0830 

15.0091 

-6.9261 

27 

9.6300 

13.4979 

-3.8679 

28 

10.9330 

11.9866 

-1.0536 

29 

5.0660 

10.4754 

-6.4094 

30 

8.2000 

8.9641 

-0.7641 

31 

11.7500 

7.4529 

4.2971 

32 

7.4500 

6.9417 

1.5083 

33 

10.4330 

4.4304 

6.0026 

34 

8.2170 

2.9192 

6.2978 

35 

4.1670 

1.4080 

2.7590 

36 

41.5000 

18.3212 

23.1788 

37 

4.5830 

16.8100 

-12.2270 

38 

17.4170 

15.2988 

2.1182 

39 

4.3670 

13.7875 

-9.4205 

40 

17.3830 

12.2763 

5.1067 

41 

5.5830 

10.7651 

-5.1821 

42 

9.9170 

9.2638 

0.6632 

43 

3.0000 

7.7426 

-4.7426 

44 

3.3300 

6.2313 

-2.9013 

45 

6.2830 

4.7201 

1.5629 

46 

28.0000 

20.6662 

7.3338 

47 

21.4170 

19.1550 

2.2620 

48 

4.7600 

17.6437 

-12.8937 
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49 

15.6670 

16.1325 

-0.4665 

50 

8.8330 

14.6212 

-5.7882 

51 

20.5000 

13.1100 

7.3900 

52 

6.7500 

11.5988 

-5.8488 

53 

8.2500 

10.0875 

-1.8375 

64 

13.7500 

8.5763 

5.1737 

55 

13.5830 

7.0651 

6.5179 

56 

14.2330 

14.6639 

-0.4309 

57 

9.5000 

13.1627 

-3.6527 

58 

6.3330 

11.6414 

-5.3084 

59 

17.2600 

10.1302 

7.1198 

60 

5.8330 

8.6189 

-2.7859 

61 

7.8330 

7.1077 

0.7253 

62 

4.4170 

5.5965 

-1.1795 

63 

3.2500 

4.0852 

-0.8352 

64 

7.5000 

2 . 5740 

4.9260 

65 

4.4670 

1.0628 

3.4042 

66 

11.5000 

13.2305 

-1.7305 

67 

11.4170 

11.7193 

-0.3023 

68 

6.8330 

10.2081 

-3.3751 

69 

7.0830 

8.6968 

-1.6138 

70 

5.2500 

7 . 1856 

-1.9356 

71 

6.9170 

5.6744 

1.2426 

72 

4.5830 

4.1631 

0.4199 

73 

2.7170 

2.6519 

0.0651 

74 

2.4500 

1 . 1406 

1 . 3094 

75 

3.5670 

-0.3706 

3.9376 

Sum  of  Residuals  0 
Sum  of  Squared  Residuals  2777.0149 
Predicted  Resid  SS  (Press)  3482.4609 
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Dep  Var 

Predict 

Std  Err 

Std  Err 

Student 

Obs 

TIME 

Value 

Predict 

Residual 

Residual 

Residual 

1 

39.5830 

21.0134 

1.989 

18.5696 

6.073 

3.058 

2 

21.1670 

19.5022 

1.873 

1 . 6648 

6.110 

0.272 

3 

12.7500 

17.9909 

1.787 

-5.2409 

6.136 

-0.854 

4 

12.1670 

16.4797 

1.736 

-4.3127 

6.150 

-0.701 

5 

14.0830 

14.9684 

1.724 

-0.8864 

6.154 

-0.144 

6 

4.7500 

13.4572 

1.761 

-8.7072 

6.146 

-1.417 

7 

12.2500 

11.9460 

1.814 

0.3040 

6.128 

0.050 

8 

7.7500 

10.4347 

1.912 

-2.6847 

6.098 

-0.440 

9 

12.0830 

8.9235 

2.037 

3.1595 

6.057 

0.522 

10 

7.2500 

7.4123 

2.187 

-0 . 1623 

6.005 

-0.027 

11 

17.0830 

19.5800 

2.116 

-2.4970 

6.030 

-0.414 

12 

15.1670 

18.0688 

2.022 

-2.9018 

6.062 

-0.479 

13 

16.8330 

16.6576 

1.959 

0.2754 

6.083 

0.045 

14 

20.7000 

15.0463 

1.929 

5.6537 

6.092 

0.928 

16 

11.3000 

13.5351 

1.934 

-2.2351 

6.091 

-0.367 

16 

40.4160 

20.2207 

2.265 

20.1953 

5.976 

3.380 

17 

24.8330 

18.7095 

2.135 

6.1235 

6.023 

1.017 

18 

9.6670 

17.1983 

2.030 

-7.5313 

6.059 

-1.243 

19 

11.5000 

15.6870 

1.955 

-4.1870 

6.084 

-0.688 

20 

8.9500 

14.1758 

1.912 

-5 . 2268 

6.098 

-0.857 

21 

8.1600 

12.6646 

1.904 

-4.5046 

6.100 

-0.738 

22 

8.5000 

11.1533 

1.931 

-2.6533 

6.092 

-0.436 

23 

6.4160 

9.6421 

1.992 

-3.2261 

6.072 

-0.531 

24 

8.4160 

8.1308 

2.084 

0.2852 

6.041 

0.047 

25 

5.5000 

6.6196 

2.203 

-1.1196 

5.999 

-0.187 

26 

8.0830 

15.0091 

2.203 

-6.9261 

5.999 

-1.155 

27 

9.6300 

13.4979 

2.084 

-3.8679 

6.041 

-0.640 

28 

10.9330 

11.9866 

1.992 

-1.0536 

6.072 

-0.174 

29 

5.0660 

10.4764 

1.931 

-5.4094 

6.092 

-0.888 

30 

8 . 2000 

8.9641 

1.904 

-0.7641 

6.100 

-0.125 

31 

11.7500 

7.4529 

1.912 

4.2971 

6.098 

0.705 

32 

7.4500 

5.9417 

1.955 

1.5083 

6.084 

0.248 

33 

10.4330 

4.4304 

2.030 

6.0026 

6.059 

0.991 

34 

8.2170 

2.9192 

2.135 

5.2978 

6.023 

0.880 

35 

4.1670 

1.4080 

2.265 

2.7590 

5.976 

0.462 

36 

41.5000 

18.3212 

2.265 

23.1788 

5.976 

3.879 

37 

4.5830 

16.8100 

2.135 

-12.2270 

6.023 

-2.030 

38 

17.4170 

15.2988 

2.030 

2.1182 

6.059 

0.350 

39 

4.3670 

13.7875 

1.955 

-9.4205 

6.084 

-1.548 

40 

17.3830 

12.2763 

1.912 

5.1067 

6.098 

0.837 

41 

5.5830 

10.7651 

1.904 

-5.1821 

6.100 

-0.849 

42 

9.9170 

9.2538 

1.931 

0.6632 

6.092 

0.109 

43 

3.0000 

7.7426 

1.992 

-4.7426 

6.072 

-0.781 

44 

3.3300 

6.2313 

2.084 

-2.9013 

6.041 

-0.480 

45 

6.2830 

4.7201 

2.203 

1.5629 

5.999 

0.261 

46 

28.0000 

20.6662 

2.203 

7.3338 

5.999 

1.223 

47 

21.4170 

19.1550 

2.084 

2.2620 

6.041 

0.374 

48 

4.7500 

17.6437 

1.992 

-12.8937 

6.072 

-2.123 
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49 

15.6670 

16.1325 

1.931 

-0.4655 

6.002 

-0.076 

GO 

8.0330 

14.6212 

1 . 004 

-5.7882 

6.100 

-0.049 

51 

20.5000 

13.1100 

1.012 

7.3900 

6.098 

1.212 

52 

5.7600 

11.5988 

1.955 

-5.8488 

6.084 

-0.961 

53 

8.2500 

10.0875 

2.030 

-1.8375 

6.059 

-0.303 

54 

13.7500 

8.5763 

2.135 

5.1737 

6.023 

0.859 

55 

13.5830 

7.0651 

2.265 

6.5179 

B.97G 

1.091 

56 

14.2330 

14.6639 

2.027 

-0.4309 

G.061 

-0.071 

57 

9 . 6000 

13.1527 

1.881 

-3.6527 

6.107 

-0.698 

53 

6 . 3330 

11.6414 

1.761 

-5.3084 

6. 143 

-0 .864 

59 

17.2500 

10.1302 

1.673 

7.1198 

6.168 

1.154 

60 

5.8330 

8.6189 

1.623 

-2.7859 

6.181 

-0.451 

61 

7.8330 

7.1077 

1.614 

0.7253 

6.183 

0.117 

62 

4.4170 

5.6965 

1.646 

-1.1795 

6.175 

-0.191 

63 

3.2500 

4.0852 

1.717 

-0.8352 

6.156 

-0.136 

64 

7.5000 

2 . 5740 

1.823 

4.9260 

6.125 

0.804 

65 

4.4670 

1.0628 

1.958 

3.4042 

6.083 

0.560 

66 

11.5000 

13.2305 

1.958 

-1.7305 

6.083 

-0.284 

67 

11.4170 

11.7193 

1.823 

-0.3023 

6.125 

-0.049 

68 

6.8330 

10.2081 

1.717 

-3.3751 

6.156 

-0.548 

69 

7.0830 

8.6968 

1.646  - 

-1.6138 

6.175 

-0.261 

7C 

5.2500 

7.1856 

1.614 

-1.9356 

6.183 

-0.313 

71 

6.9170 

5 . 6744 

1.623 

1 . 2426 

6.181 

0.201 

72 

4.5830 

4.1631 

1.673 

0.4199 

6.168 

0.068 

73 

2.7170 

2.6519 

1.761 

0.0651 

6.143 

0.011 

74 

2.4500 

1 . 1406 

1.881 

1 . 3094 

6.107 

0.214 

75 

3.5670 

-0.3706 

2.027 

3.9376 

6.061 

0.650 

Cook's 

Obs  -2-1-0  12  D 


1  1 

| ****** 

0.143 

2  1 

1 

0.001 

3  I 

*| 

0.009 

4  I 

*| 

0.006 

5  I 

1 

0.000 

6  I 

**l 

0.023 

7  I 

1 

0.000 

8  I 

1 

0.003 

9  I 

1* 

0.004 

10  I 

1 

0.000 

11  1 

1 

0.003 

12  1 

1 

0.004 

13  1 

1 

0.000 

14  I 

|  * 

0.012 

15  I 

1 

0.002 

16  I 

|  ****** 

0.234 

17  I 

|** 

0.019 

18  1 

**| 

0.025 

19  1 

*  | 

0.007 

20  I 

*  | 

0.010 

21  1 

*  | 

0.008 

22  1 

1 

0.003 

23  I 

*  | 

0.004 

24  1 

1 

0.000 

25  1 

0.001 

26  I 

**  1 

0.026 

27  I 

*| 

0.007 

28  1 

1 

0.000 

29  i 

*  | 

0.011 

30  1 

1 

0.000 

31  1 

|  * 

0.007 

32  1 

1 

0.001 

33  1 

|  * 

0.016 

34  1 

1* 

0.014 

35  I 

0.004 

36  1 

|  ****** 

0.309 

37  1 

****  | 

0.074 

38  1 

1 

0.002 

39  1 

***  | 

0.035 

40  1 

|* 

0.010 

41  1 

*  | 

0.010 

42  1 

1 

0.000 

43  | 

*  | 

0.009 

44  1 

1 

0.004 

45  1 

1 

0.001 

46  I 

I** 

0.029 

47  I 

1 

0.002 

48  1 

****  | 

0.069 
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46  I 

1 

0.000 

SO  I 

*| 

0.013 

SI  1 

1** 

0.021 

62  1 

*  | 

0.014 

S3  1 

1 

0.001 

64  I 

|* 

0.013 

SS  i 

1** 

0.024 

56  | 

1 

0.000 

57  I 

*  | 

O.OOS 

58  I 

*| 

0.009 

69  I 

1** 

0.014 

60  I 

1 

0.002 

61  I 

1 

0.000 

62  I 

1 

0.000 

63  I 

1 

0.000 

64  1 

|  * 

0.008 

65  1 

0.005 

66  I 

1 

0.001 

67  I 

1 

0.000 

68  I 

*  | 

0.003 

69  1 

1 

0.001 

70  I 

1 

0.001 

71  I 

1 

0.000 

72  I 

1 

0.000 

73  1 

1 

0.000 

74  1 

1 

0.001 

75  I 

|* 

0.007 

Sum  of  Residuals  0 
Sum  of  Squared  Residuals  2777.0149 
Predicted  Resid  SS  (Press)  3482.4609 
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EXPERIMEIT  2:  FACILITATED-MAIUAL  VERSUS  MAIUAL 
TEST  FOR  IORMALITY  OF  RESIDUALS 
RU1  EXP2ARSD 


Univariate  Procedure 


Variable=RESIDUAL 


Moments 


H 

7S 

Sum  Wgts 

75 

Mean 

-1.33E-6 

Sum 

-0.0001 

Std  Dev 

6.125949 

Variance 

37.52726 

Skewness 

1.323592 

Kurtosis 

3.841498 

USS 

2777.017 

CSS 

2777.017 

CV 

-4.594E8 

Std  Mean 

0.707364 

T : Mean=0 

-1.88E-6 

Prob>|T| 

1.0000 

Mum  "=  0 

75 

Vum  >  0 

33 

M(Sign) 

-4.5 

Prob> I M 1 

0 . 3557 

Sgn  Rank 

-169 

Prob> I S 1 

0.4048 

W: Normal 

0.907471 

ProbCW 

0.0001 

quantiles (Dei-5) 

1007.  Max 

23.1788 

997. 

23.1788 

757.  q3 

2.759 

95% 

7.39 

507.  Med 

-0.7641 

90% 

6 . 1235 

257.  qi 

-3.6627 

10% 

-5.7882 

07.  Min 

-12.8937 

5% 

-8.7072 

Range 

q3-qi 

Mode 

36.0725 

6.4117 

-12.8937 

1% 

-12.8937 

Extremes 


Lowest 

Obs 

Highest 

□bs 

-12.8937( 

48) 

7.3338C 

46) 

-12. 227 ( 

37) 

7.39( 

51) 

-9. 4205 ( 

39) 

18. 5696 ( 

1) 

-8. 7072 ( 

6) 

20. 1953 ( 

16) 

-7.5313( 

18) 

23.1788C 

36) 
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Univariate  Procedure 


Variable=RESIDUAL 


Moments 


K 

75 

Sum  Vgts 

76 

Kean 

-1.33E-6 

Sum 

-0.0001 

Std  Dev 

6.125949 

Variance 

37.52725 

Skevness 

1.323592 

Kurtosis 

3.841498 

USS 

2777.017 

CSS 

2777.017 

CV 

-4.594E8 

Std  Mean 

0.707364 

T : Mean=0 

-1.88E-6 

Prob> I T I 

1.0000 

Mum  “=  0 

75 

Hum  >  0 

33 

M(Sign) 

-4.5 

Prob>|Ml 

0.3657 

Sgn  Rank 

-159 

Prob> I S I 

0.4048 

Quantiles (Del =5) 


1007.  Max 

23.1788 

997. 

23.1788 

757.  q3 

2.759 

957. 

7.39 

507.  Med 

-0.7641 

907. 

6.1235 

257.  Q1 

-3.6527 

107. 

-5.7882 

07.  Min 

-12.8937 

57. 

-8.7072 

Range 

q3-qi 

Mode 

36.0725 

6.4117 

-12.8937 

17. 

-12.8937 

Extremes 


Lovest 

Obs 

Highest 

Obs 

-12. 8937 ( 

48) 

7 . 3338 ( 

46) 

-12. 227 ( 

37) 

7. 39( 

51) 

-9. 4205 ( 

39) 

18.5696C 

1) 

-8. 7072 ( 

6) 

20 . 1953( 

16) 

-7.5313( 

18) 

23. 1788 ( 

36) 
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Univariate  Procedure 


Variable=RESIDUAL 


Stem  Leal 

# 

Bozplot 

22  2 

1 

* 

20  2 

1 

0 

18  6 

1 

0 

16 

14 

12 

10 

8 

6  015134 

6 

1 

4  391237 

6 

1 

2  138249 

6 

+ - + 

0  133347723567 

12 

1  1 

-0  98762119885432 

14 

* — + — * 

-2  97429987752 

11 

+ - + 

-4  88432227532 

11 

1 

-6  59 

2 

1 

-8  47 

2 

1 

-10 

1 

-12  92 

2 

1 

- + - + - + — 

— + 

lormal  Probability  Plot 

23+  * 

I  * 

I  * 


5+ 


+++ 

+++ 

+++ 

+++ 

++***♦  * 

++*** 

++*** 

****** 

***** 

***** 

******** 

**++++ 

***** 

+++ 


-13+  *  ++* 


+ - + - +. 


-2 


._+ - + 

+2 
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EXPERIMENT  2:  FACILITATED-MANUAL  VERSUS  MAIUAL 
REGRESSION  AVALYSIS  USING  IVDICATOR  VARIABLES 
WITH  RESIDUAL  PLOT  AID  TIME  TRAISFORMED  USIIG  (TIME)* (.6) 


Model:  MODEL1 
Dependent  Variable:  TIME 

Analysis  of  Variance 


Sum 

of  Mean 

Source 

DF  Squares  Square 

F  Value 

Prob>F 

Model 

6  40.67635  6.76266 

10. 

.807 

0.0001 

Error 

68  42.56160  0.62576 

C  Total 

74  83.12695 

Root  MSE 

0.79105 

R- square 

0.4881 

Dep  Mean 

3.17253 

Adj  R-sq 

0.4429 

C.V 

• 

24.93431 

Parameter  Estimates 

Parameter 

Standard  T  : 

for  HO: 

Variable 

DF 

Estimate 

Error  Parameter^ 

Prob  >  iTl 

IHTERCEP 

1 

3.756417 

0.27161308 

13.826 

0.0001 

GR0UP4 

1 

0.417847 

0.37065433 

1.127 

0.2636 

GROUPS 

1 

0.283274 

0.27967811 

1.013 

0.3147 

GROUPS 

1 

0.471522 

0.27967811 

1.686 

0.0964 

DAY 

1 

-0.115886 

0.18532716 

-0.625 

0.5339 

INTERVAL 

1 

-0.202222 

0.03217281 

-6.286 

0.0001 

FACILITY 

1 

0.567671 

0.25015171 

2.269 

0.0265 

Variance 

Variable 

DF 

Inflation 

IITERCEP 

1 

0.00000000 

GR0UP4 

1 

2.63458647 

GROUPS 

1 

1.83333333 

GROUP 6 

1 

1.83333333 

DAY 

1 

1.02456140 

INTERVAL 

1 

1.02005013 

FACILITY 

1 

1.86666667 

Durbin-Vatson 

D 

2.219 

(For  lumber  of  Obs.)  76 

1st  Order  Autocorrelation  -0.146 
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Dap  V&r 

Predict 

Obs 

TINE 

Value 

1 

6.2915 

4.5386 

2 

4.6008 

4.3364 

3 

3.5707 

4.1342 

4 

3.4881 

3.9319 

5 

3.7627 

3.7297 

6 

2.1794 

3.5276 

7 

3.5000 

3.3253 

8 

2.7839 

3.1231 

9 

3.4761 

2.9208 

10 

2.6926 

2.7186 

11 

4.1332 

4.4227 

12 

3.8945 

4.2205 

13 

4.1028 

4.0183 

14 

4.5497 

3.8161 

15 

3.3615 

3.6138 

16 

6.3574 

4.4040 

17 

4.9833 

4.2018 

18 

3.1092 

3.9996 

19 

3.3912 

3.7974 

20 

2.9917 

3.5952 

21 

2.8566 

3.3929 

22 

2.9155 

3.1907 

23 

2.5330 

2.9885 

24 

2.9010 

2.7863 

25 

2.3452 

2.5840 

26 

2.8431 

3.7206 

27 

3.1032 

3.5184 

28 

3.3065 

3.3161 

29 

2.2508 

3.1139 

30 

2.8636 

2.9117 

31 

3.4278 

2.7095 

32 

2.7295 

2.5072 

33 

3.2300 

2.3050 

34 

2.8665 

2.1028 

35 

2.0413 

1.9006 

36 

6.4420 

4.0247 

37 

2 . 1408 

3.8225 

38 

4.1734 

3.6203 

39 

2.0897 

3.4180 

40 

4.1693 

3.2158 

41 

2.3628 

3.0136 

42 

3.1491 

2.8114 

43 

1.7321 

2.6092 

44 

1.8248 

2.4069 

45 

2.5066 

2.2047 

46 

5.2915 

4.4764 

47 

4.6279 

4.2742 

48 

2.1794 

4.0720 

F-58 


Residual 

1.7529 
0.2644 
-0.5635 
-0.4438 
0.0230 
-1.3481 
0.1747 
-0.3392 
0 . 5552 
-0.0260 
-0.2896 
-0.3260 
0.0845 
0.7337 
-0.2523 
1.9533 
0.7815 
-0.8904 
-0.4062 
-0.6035 
-0.5364 
-0.2752 
-0.4555 
0.1148 
-0.2388 
-0.8775 
-0.4151 
-0.00963 
-0.8631 
-0.0481 
0.7184 
0.2222 
0.9250 
0.7637 
0.1407 
2.4173 
-1.6817 
0.5531 
-1.3283 
0.9535 
-0.6508 
0.3377 
-0.8771 
-0.6821 
0.3019 
0.8151 
0.3537 
-1.8925 


49 

3.9582 

3.8697 

0.0884 

50 

2.9720 

3.6675 

-0.6955 

51 

4.5277 

3.4653 

1.0624 

52 

2.3979 

3.2631 

-0.8652 

53 

2.8723 

3.0608 

-0.1886 

54 

3.7081 

2.8586 

0.8495 

55 

3.6855 

2.6564 

1.0291 

56 

3.7727 

3.5532 

0.2195 

57 

3.0822 

3.3510 

-0.2688 

58 

2.5165 

3.1487 

-0.6322 

59 

4.1533 

2.9465 

1.2068 

60 

2.4152 

2.7443 

-0.3291 

61 

2.7987 

2.5421 

0.2567 

62 

2.1017 

2.3399 

-0.2382 

63 

1.8028 

2.1376 

-0 . 3349 

64 

2.7386 

1.9354 

0.8032 

65 

2.1135 

1 . 7332 

0.3803 

66 

3.3912 

3.4373 

-0.0461 

67 

3.3789 

3.2351 

0.1438 

68 

2.6140 

3.0329 

-0.4189 

69 

2.6614 

2.8306 

-0.1693 

70 

2.2913 

2.6284 

-0.3371 

71 

2.6300 

2.4262 

0.2038 

72 

2.1408 

2.2240 

-0.0832 

73 

1.6483 

2.0218 

-0.3734 

74 

1.5652 

1.8195 

-0.2543 

75 

1.8887 

1.6173 

0.2713 

Sum  of  Residuals  0 
Sum  of  Squared  Residuals  42.5S16 
Predicted  Resid  SS  (Press)  52.9027 
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RESIDUAL 


2.0  2.5  3.0  3.5  4.0  4.5 

Predicted  Value  of  TINE  PRED 
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Dep  Var 

Predict 

Std  Err 

Obs 

TIME 

Value 

Predict 

1 

6.2915 

4.5386 

0.246 

2 

4.6008 

4.3364 

0.232 

3 

3.5707 

4.1342 

0.221 

4 

3.4881 

3.9319 

0.215 

S 

3.7527 

3.7297 

0.213 

6 

2.1794 

3.5275 

0.217 

7 

3.5000 

3.3253 

0.225 

8 

2.7839 

3.1231 

0.237 

9 

3.4761 

2.9208 

0.252 

10 

2.6926 

2.7186 

0.271 

11 

4.1332 

4.4227 

0.262 

12 

3.8945 

4.2205 

0.250 

13 

4.1028 

4.0183 

0.243 

14 

4.5497 

3.8161 

0.239 

15 

3.3615 

3.6138 

0.239 

16 

6.3574 

4.4040 

0.280 

17 

4.9833 

4.2018 

0.264 

18 

3.1092 

3.9996 

0.251 

19 

3.3912 

3.7974 

0.242 

20 

2.9917 

3.5952 

0.237 

21 

2.8566 

3.3929 

0.236 

22 

2.9155 

3.1907 

0.239 

23 

2.5330 

2.9885 

0.247 

24 

2.9010 

2.7863 

0.258 

25 

2.3462 

2.5840 

0.273 

26 

2.8431 

3.7206 

0.273 

27 

3.1032 

3.6184 

0.258 

28 

3.3066 

3.3161 

0.247 

29 

2.2508 

3.1139 

0.239 

30 

2.8636 

2.9117 

0.236 

31 

3.4278 

2.7096 

0.237 

32 

2.7295 

2.5072 

0.242 

33 

3.2300 

2.3050 

0.251 

34 

2.8665 

2.1028 

0.264 

35 

2.0413 

1.9006 

0.280 

36 

6.4420 

4.0247 

0.280 

37 

2.1408 

3.8225 

0.264 

38 

4.1734 

3.6203 

0.251 

39 

2.0897 

3.4180 

0.242 

40 

4.1693 

3.2158 

0.237 

41 

2.3628 

3.0136 

0.236 

42 

3.1491 

2.8114 

0.239 

43 

1.7321 

2.6092 

0.247 

44 

1.8248 

2.4069 

0.258 

45 

2.5066 

2.2047 

0.273 

46 

5.2915 

4.4764 

0.273 

47 

4.6279 

4.2742 

0.258 

48 

2.1794 

4.0720 

0.247 

Std  Err  Student 
Residual  Residual  Residual 


1.7529 

0.752 

2.332 

0.2644 

0.756 

0.350 

-0.5635 

0.759 

-0.742 

-0.4438 

0.761 

-0.683 

0.0230 

0.762 

0.030 

-1.3481 

0.761 

-1.772 

0.1747 

0.758 

0.230 

-0.3392 

0.755 

-0.449 

0 . 5552 

0.750 

0.741 

-0.0260 

0.743 

-0.036 

-0.2896 

0.746 

-0.388 

-0.3260 

0.750 

-0.434 

0 . 0845 

0.753 

0.112 

0.7337 

0.754 

0.973 

-0.2523 

0.764 

-0.335 

1.9533 

0.740 

2.641 

0.7815 

0.746 

1.048 

-0.8904 

0.750 

-1.187 

-0.4062 

0.763 

-0.639 

-0.6035 

0.755 

-0.800 

-0.5364 

0.755 

-0.710 

-0.2752 

0.754 

-0.365 

-0.4555 

0.752 

-0.606 

0.1148 

0.748 

0.153 

-0.2388 

0.743 

-0.322 

-0.8775 

0.743 

-1.182 

-0.4151 

0.748 

-0.555 

-0.00963 

0.762 

-0.013 

-0.8631 

0.754 

-1.145 

-0.0481 

0.755 

-0.064 

0.7184 

0.755 

0.952 

0.2222 

0.753 

0.295 

0.9250 

0.750 

1.233 

0.7637 

0.746 

1.024 

0 . 1407 

0.740 

0.190 

2.4173 

0.740 

3.268 

-1.6817 

0.746 

-2.256 

0.5531 

0.750 

0.737 

-1.3283 

0.753 

-1.764 

0.9535 

0.755 

1.263 

-0.6508 

0.755 

-0.862 

0.3377 

0.754 

0.448 

-0.8771 

0.752 

-1.167 

-0.5821 

0.748 

-0.778 

0.3019 

0.743 

0.407 

0.8151 

0.743 

1.098 

0.3537 

0.748 

0.473 

-1.8925 

0.752 

-2.518 
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48 

3.9582 

3.8697 

0.236 

0.0884 

0.754 

0.117 

50 

2.9720 

3.6675 

0.236 

-0.6955 

0.755 

-0.921 

51 

4.5277 

3.4653 

0.237 

1.0624 

0.755 

1.408 

52 

2.3979 

3.2631 

0.242 

-0.8652 

0.763 

-1.149 

53 

2.8723 

3.0608 

0.251 

-0.1886 

0.760 

-0.251 

54 

3.7081 

2.8586 

0.264 

0.8495 

0.746 

1.139 

65 

3.6855 

2.6564 

0.280 

1.0291 

0.740 

1.391 

56 

3.7727 

3.5532 

0.251 

0.2195 

0.750 

0.293 

57 

3.0822 

3.3510 

0.233 

-0.2688 

0.756 

-0.356 

58 

2.5165 

3.1487 

0.218 

-0.6322 

0.760 

-0.831 

59 

4.1533 

2.9465 

0.207 

1.2068 

0.763 

1.581 

60 

2.4152 

2.7443 

0.201 

-0.3291 

0.765 

-0.430 

61 

2.7987 

2.5421 

0.200 

0.2567 

0.765 

0.335 

62 

2.1017 

2.3396 

0.204 

-0.2382 

0.764 

-0.312 

63 

1.8028 

2.1376 

0.213 

-0.3349 

0.762 

-0.439 

64 

2.7386 

1.9354 

0.226 

0.8032 

0.758 

1.059 

65 

2.1135 

1.7332 

0.242 

0.3803 

0.763 

0.505 

66 

3.3912 

3.4373 

0.242 

-0.0461 

0.753 

-0.061 

67 

3.3789 

3.2351 

0.226 

0.1438 

0.758 

0.190 

68 

2.6140 

3.0329 

0.213 

-0.4189 

0.762 

-0.550 

69 

2.6614 

2.8306 

0.204 

-0.1693 

0.764 

-0.221 

70 

2.2913 

2.6284 

0.200 

-0.3371 

0.765 

-0.440 

71 

2.6300 

2.4262 

0.201 

0.2038 

0.765 

0.266 

72 

2.1408 

2.2240 

0.207 

-0.0832 

0.763 

-0.109 

73 

1.6483 

2.0218 

0.218 

-0.3734 

0.760 

-0.491 

74 

1.5652 

1.8195 

0.233 

-0.2543 

0.756 

-0.336 

75 

1 . 8887 

1.6173 

0.251 

0.2713 

0.750 

0.362 
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Cook ’ 8 

Obs  -2-1-0  12  D 


1  1 

|**** 

0.083 

2  I 

1 

0.002 

3  I 

*  | 

0.007 

4  I 

*  | 

0.004 

5  I 

1 

0.000 

6  I 

***  | 

0.036 

7  I 

1 

0.001 

8  I 

1 

0.003 

9  | 

1  * 

0.009 

10  | 

1 

0.000 

11  1 

1 

0.003 

12  | 

1 

0.003 

13  | 

1 

0.000 

14  | 

|* 

0.014 

15  I 

1 

0.002 

16  I 

|  ***** 

0.143 

17  I 

1  ** 

0.020 

18  | 

**  | 

0.023 

19  | 

*  1 

0.004 

20  | 

*1 

0.009 

21  I 

*  1 

0.007 

22  I 

1 

0.002 

23  ! 

★  | 

0.006 

24  I 

1 

0.000 

25  | 

1 

0.002 

26  | 

**| 

0.027 

27  I 

*  | 

0.005 

28  | 

1 

0.000 

29  | 

**  | 

0.019 

30  | 

1 

0.000 

31  I 

|  * 

0.013 

32  I 

1 

0.001 

33  I 

|** 

0.024 

34  I 

|  ** 

0.019 

35  I 

1 

0.001 

36  | 

| ****** 

0.219 

37  | 

****  | 

0.091 

38  | 

1  * 

0.009 

39  | 

***  | 

0.046 

40  f 

|  ** 

0.022 

41  I 

*  1 

0.010 

42  | 

1 

0.003 

43  | 

**  | 

0.021 

44  | 

*  1 

0.010 

45  | 

1 

0.003 

46  | 

|  ** 

0.023 

47  | 

1 

0.004 

48  | 

***** | 

0.097 
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49  | 

1 

0.000 

50  | 

0.012 

51  | 

|** 

0.028 

52  | 

**| 

0.019 

53  | 

1 

0.001 

54  | 

0.023 

55  | 

|** 

0.040 

56  | 

1 

0.001 

67  | 

1 

0.002 

58  | 

*  | 

0.008 

59  | 

|*** 

0.026 

60  | 

1 

0.002 

61  I 

1 

0.001 

62  | 

1 

0.001 

63  | 

1 

0.002 

64  | 

|** 

0.014 

65  | 

|* 

0.004 

66  | 

1 

0.000 

67  | 

I 

0.000 

68  | 

*  | 

0.003 

69  | 

1 

0.000 

70  | 

I 

0.002 

71  I 

1 

0.001 

72  | 

1 

0.000 

73  | 

1 

0.003 

74  | 

1 

0.002 

76  | 

1 

0.002 

Sum  of  Residuals 

Sum  of  Squared  Residuals 

Predicted  Resid  SS  (Press) 


0 

42.5516 

52.9027 


EXPERIMENT  2:  FACILITATED-MANUAL  VERSUS  MANUAL 
TEST  FOR  IORMALITY  OF  RESIDUALS 
RU1  EXP2WRSD  TRANSFORMED 


Univariate  Procedure 


Variable=RESIDUAL 


Moments 


I 

75 

Sum  Wgts 

75 

Mean 

-1.73E-6 

Sum 

-0.00013 

Std  Dev 

0.758303 

Variance 

0.575024 

Skewness 

0.464149 

Kurtosis 

1.244352 

USS 

42.55177 

CSS 

42.55177 

CV 

-4.375E7 

Std  Mean 

0.087561 

T : Mean=0 

-0.00002 

Prob>|T| 

1.0000 

Nun  *=  0 

75 

Nun  >  0 

34 

M(Sign) 

-3.5 

Prob>|M| 

0.4887 

Sgn  Rank 

-89 

Prob> I S I 

0.6415 

V : Normal 

0.973672 

Prob<W 

0.3413 

Quantiles (Def =5) 


100‘/.  Max 

2.4173 

99% 

2.4173 

75*/.  q3 

0.3637 

95% 

1.2068 

50'/.  Med 

-0.0481 

90% 

0.925 

25'/.  Q1 

-0.4189 

10% 

-0.8652 

O'/.  Min 

-1 . 8925 

5% 

-1.3283 

Range 

Q3-Q1 

Mode 

4.3098 

0.7726 

-1.8925 

1% 

-1.8925 

Extremes 


Lowest 

Obs 

Highest 

Obs 

-1.8925( 

48) 

1 . 0624( 

51) 

-1.6817( 

37) 

1 . 2068 ( 

59) 

-1.348K 

6) 

1.7529C 

1) 

-1.3283( 

39) 

1 . 9533  ( 

16) 

-0.8904( 

18) 

2.4173( 

36) 
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Univariate  Procedure 


Variable=RESIDUAL 


Moments 


I 

75 

Sum  Vgts 

75 

Mean 

-1.73E-6 

Sum 

-0.00013 

Std  Dev 

0.758303 

Variance 

0.575024 

Skevness 

0.464149 

Kurtosis 

1.244352 

USS 

42.55177 

CSS 

42.55177 

CV 

-4.376E7 

Std  Mean 

0.087561 

T:Mean=0 

-0.00002 

Prob> | T I 

1.0000 

Mum  "=  0 

75 

Mum  >  0 

34 

M(Sign) 

-3.5 

Prob> I M 1 

0.4887 

Sgn  Rank 

-89 

Prob> | S 1 

0.6415 

quantiles (Del =5) 


100'/.  Max 

2.4173 

997. 

2.4173 

75'/.  q3 

0.3537 

95% 

1.2068 

50'/.  Med 

-0.0481 

907. 

0.925 

267.  qi 

-0.4189 

107. 

-0.8652 

07.  Min 

-1.8925 

57. 

-1.3283 

Range 

Q3-qi 

Mode 

4.3098 

0.7726 

-1.8925 

17. 

-1.8925 

Extremes 


Lowest 

Obs 

Highest 

Obs 

-1.8925( 

48) 

1.0624C 

51) 

-1.6817C 

37) 

1 . 2068 ( 

59) 

-1.3481( 

6) 

1.7529( 

1) 

-1.3283( 

39) 

1.9533C 

16) 

-0.8904( 

18) 

2.4173C 

36) 
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Univariate  Procedure 


Vaxiable=RESIDUAL 


Stem  Leal  # 

24  2  1 

22 
20 

18  5  1 

16  5  1 

14 

12  1  1 

10  36  2 

8  02S2S  B 

6  2368  4 

4  56  2 

2  02266704S8  10 

0  2891447  7 

-0  9785531  7 

-2  7443339875544  13 

-4  86464221  8 

-6  0530  4 

-8  98876  5 

-10 

-12  53  2 

-14 

-16  8  1 

-18  9  1 

- + - + - + - + 


Multiply  Stem. Leaf  by  10**-1 


Bozplot 

0 


0 

0 
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Univariate  Procedure 


Variable=RESIDUAL 


2.5+ 


lomal  Probability  Plot 

* 


0.3+ 


*  ++ 
*  ++ 
+++ 

+*+ 

** 

**** 

++* 

++* 

+*♦** 

*** 

+*** 

****** 

**** 

**++ 

****+ 

++ 

**+ 

+++ 

++  * 


-1.9++* 

+ - + - + - + - + - + - + - + - + - + - + 

-2  -1  0  +1  +2 
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Appendix  G.  Consensus  Data 
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Figure  G.l.  Consensus  Recordings  for  Group  1,  Day  1,  Career  Field  1. 
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DAY:  2 


FACILITATOR; 


CAREER  FIELD:  Manufacturing  Engrtaar 


TRAIN  IRQ  TASK  NUMBER 

LEVEL  1 _ 2  3  4 _ 5 _ 6 _ 7  B  9  10 


a 
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INTERVAL  TIME: 
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TRAINING 

LEVEL 


TASK  NUMBER 


INTERVAL  TIME: 

13.930 


TRAINING 

LEVEL 
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2 

a 

4 

a 


TASK  NUMBER 


INTERVAL  TIME: 
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Figure  G.2.  Consensus  Recordings  for  Group  1,  Day  2,  Career  Field  3. 


G-3 


DAV:  2 


FACSJTATOR: 


I  Ell 
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Figure  G.5.  Consensus  Recordings  for  Group  2,  Day  1,  Career  Field  2. 
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Figure  G.7.  Consensus  Recordings  for  Group  3,  Day  1,  Career  Field  1 
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Figure  G.8.  Consensus  Recordings  for  Group  3,  Day  1,  Career  Field  2. 


FACILITATOR:  TE 


CAREER  FELO:  MNnMnsncs  QA/OC 


TASK  NUMBER 

S 


INTERVAL  TIME 

11.500 


■■ 

■ 

■ 

mm 

■■ 

■ 

■ 

■■ 

bbb 

■■■ 

■■ 

■ 

■■■ 

BH 

B 

■■ 

BB 

■ 

ia 

■■ 

n» 

■■ 

■ 

■ 

■■ 

H 

■■ 

■ 

■■ 

■■ 

■ 

■ 

■■ 

■■ 

■■■ 

■■■ 

■■ 

■ 

■ 

□ 

■ 

■■ 

■■ 

■■■ 

■■ 

■■ 

■ 

■■■ 

■■■ 

■ 

■ 

BB 

■■ 

■■ 

■■ 

■ 

INTERVAL  TIME: 

13.733 


mmm 

tt 

TT 

TT 

n 

"T~r 

TT 

"" 

5 

5 

BH 

■■■ 

B 

■■■ 

5 

■■■ 

auu 

■■■ 

■■■ 

— 

_5_ 

— 

BH 

■■■ 

■■■ 

— 

■■■ 

— 

INTERVAL  TIME: 

7.800 

TRAINING 

TASK  NUMBER 

El 

BB 

■■ 

«■ 

BB 

■ 

□□ 

■  ■ 

■■■ 

□■■■■ 

B 

BB 

■ 

■  ■ 

■■ 

■ 

oa 

■  ■ 

■ 

■ 

D 

■ 

■  ■ 

■■ 

■■■ 

■ 

mu 

B 

■ 

■■ 

Bl 

■■ 

■  nn 

■ 

■■ 

amm 

■BBBB 

B 

□B 

BB 

B 

BB 

□B 

B 

B 

B 

BB 

B 

B 

B 

B 

D 

BB 

B 

□ 

■ 

5 

B 

D 

B 

■ 

B 

□ 

BB 

B 

□ 

B 

B 

B 

B 

BB 

BB 

B 

BB 

B 

B 

BB 

B 

B 

B 

OBB 

B 

B 

B 

an 

B 

□BB 

no 

B 

B 

BB 

BB 

B 

BB 

B 

Q 

BB 

B 

B 

B 

ana 

B 

■ 

B 

B 

B 

B 

B 

BB 

a 

■ 

B 

B 

BB 

■B 

B 

BB 

B 

B 

BB 

El 

B 

B 

a 

BB 

B 

■ 

B 

B 

B 

B 

fl 

BB 

B 

■ 

B 

B 

BB 

G-10 


GROUP:  3 


DAY:  2 


FACILITATOR:  TE 


CAREER  FIELD:  Cematan  Control 


TRAINING 


TASK  NUMBER 

5 


ini 


ini 


INTERVAL  TIME: 

3.833 

TRAINING 

TASK  NUMBER 

INTERVAL  TIME: 

8  850 


TRAINING 


TASK  NUMBER 


INTERVAL  TIME: 
3.183 


TRAINING 


LEVEL 


TASK  NUMBER 


□  I 


H 

■ 

■  ■IHH 

■ 

■  ■■ 

■ 

■ 

■ 

■  ■■■ 

■■■■■ 

■ 

tsmu 

□ 

□ 

■ 

■  IRH 

mum 

■ 

im 


INTERVAL  TIME: 
2.016 


Figure  G.10.  Consensus  Recordings  for  Group  3,  Day  2,  Career  Field  4. 
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Figure  G.ll.  Consensus  Recordings  for  Group  4,  Day  1,  Career  Field  1. 
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Figure  G.12.  Consensus  Recordings  for  Group  4,  Day  1,  Career  FieJd  2. 
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Figure  G.13.  Consensus  Recordings  for  Group  4,  Day  2,  Career  Field  3. 
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Figure  G.14.  Consensus  Recordings  for  Group  5,  Day  1,  Career  Field  1. 
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Figure  G.17.  Consensus  Recordings  for  Group  5,  Day  2,  Career  Field  4. 
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Figure  G.18.  Consensus  Recordings  for  Group  6,  Day  1,  Career  Field  1. 
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Figure  G.19.  Consensus  Recordings  for  Group  6,  Day  1,  Career  Field  2. 
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Figure  G.20.  Consensus  Recordings  for  Group  6,  Day  2,  Career  Field  3. 
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Figure  G.21.  Consensus  Recordings  for  Group  6,  Day  2,  Career  Field  4. 
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Figure  G.22.  Consensus  Recordings  for  Group  7,  Day  1,  Career  Field  1. 
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Figure  G.23.  Consensus  Recordings  for  Group  7,  Day  1,  Career  Field  2 
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Figure  G.24.  Consensus  Recordings  for  Group  7,  Day  2,  Career  Field  3. 
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Figure  G.25.  Consensus  Recordings  for  Group  7,  Day  2,  Career  Field  4. 
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Appendix  H.  Contingency  Table/ Chi-Square  Test  Results  for  Consensus 


H-l 


Level  of  Consensus 


Experiment  1 
Reached  Cenwiiui 


Treatment 

YES 

NO 

GDSS 

276 

283.06 

24 

18.94 

300 

Fee -Manual 

526 

518.94 

24 

31.08 

550 

802 

48 

850 

TEST  STATISTIC:  Chi-Square  (alpha  -  .05  &  df  -  1): 
CHI-SQ  (T  Value):  4.8175017 

DECISION:  Reject  Ho 

3.841 

Treatment 

Experiment  2 
Reached  Coneeneus 

YES  NO 

Fae-Manual 

526 

534.37 

24 

15.63 

550 

Manual 

397 

388.63 

3 

11.37 

400 

923 

27 

950 

TEST  STATISTIC:  Chi-Square  (alpha  -  .05  &  <H  -  1): 
CHI-SQ  (TValua):  10.9514019 

DECISION:  Reject  Ho 

3.841 

Experiment  1:  Reached  Canaan eua 

GDSS 

FAC- 

zzzmmm 

Group 

Yes 

No 

Yea 

No 

Total 

1 

30 

20 

96 

4 

150 

!  2 

47 

6 

89 

11 

T3o  “ 

3 

199 

1 

200 

4 

148 

2 

150 

5 

98 

2 

100 

6 

95 

5 

100 

Total: 

276 

24 

526 

24 

850 

EigtrJnwrt^RiMlwdCanMi^ 


Group 

!  FAC-MAN 

MAN 

UAL 

Yet 

No 

Yea 

No 

Total 

j  1 

96 

4 

100 

2 

69 

11 

100 

4 

148 

2 

150 

5 

!  98 

2 

98 

2 

200 
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95 

5 

99 

1 

200 
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200 

200 

Total: 

526 

24 

397 

3 

950 

Figure  H.l.  Experiments  1  &  2:  Level  of  Consensus. 
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Ability  to  Roach  Consensus 


Vote  Counts  to  Consensus 

Treatment  12  3  None 

300 


550 


327  407  68  48  850 

TEST  STATISTIC:  Chi-Square  (alpha  =  .05  &  df  *  3):  7.815 

CHI-SQ  (T  Value):  1 7.26787 

DECISION:  Reject  Ho 


GDSS 


Group 

1  Vote 

2  Votes 

3  Votes 

None 

Total 
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3 

14 

13 

20 

50 
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25 

20 
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50 

3 

108 

86 

5 

1 

200 

Total: 

136 

120 

20 

24 

300 

Facilitated  Manual 


Group 

1  Vote 

2  Votes 

3  Votes 

None 

Total 

1 

37 

50 

9 

4 

100 

2 

36 

48 

5 

11 

100 

4 

46 

86 

16 

2 

150 

5 

38 

49 

11 

2 

100 

6 

34 

54 

7 

5 

100 

Total: 

191 

287 

48 

24 

550 

GDSS 

136 

120 

20 

24 

115.41 

143.65 

24.00 

16.94 

Fac-Manua! 

191 

211.59 

287 

263.35 

48 

44.00 

24 

31.06 

Figure  H.2.  Experiment  1:  Number  of  Votes  to  Reach  Consensus. 
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Appendix  I.  Chi-Square/Fisher’s  Exact  Test  Results  for  Consensus 
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EXPERIMEIT  1:  OVERALL  COISE1SUS 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  COISEIS 

FACILITY  COISEIS (US  REACHED) 

Frequency  I 


Expected 

110 

1  YES 

1 

Total 

FAC-MAI 

1  24 

1  31.059 

1  526 

1  518.94 

1 

1 

550 

CDSS 

i  24 

1  16.941 

1  276 

1  283.06 

1 

1 

300 

Total 

48 

802 

850 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  COISEIS 


Statistic 


Chi-Square 

Likelihood  Ratio  Chi-Square 
Continuity  Adj.  Chi-Square 
Mantel-Haenszel  Chi-Square 
Fisher’s  Exact  Test  (Left) 
(Right) 
(2-Tail) 

Phi  Coefficient 
Contingency  Coefficient 
Cramer's  V 

Sample  Size  =  850 


DF 

Value 

Prob 

1 

4.818 

0.028 

1 

4.616 

0.032 

1 

4.159 

0.041 

1 

4.812 

0.028 

0.022 

0.990 

0.042 

-0.075 

0.075 

-0.075 

1-2 


EXPERIMEIT  1:  ■ UMBER  OF  VOTES  TO  REACH  COISEISUS 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  VOTES 

FACILITY  VOTES (  TO  REACH  COISEISUS) 

Frequency  I 

Expected  lIOIE  lOIE  I THREE  ITVO  | 

- + - + - + - + - + 

FAC-MAI  I  24  I  191  |  48  I  287  I 

I  31.059  I  211.59  I  44  I  263.35  I 

- + - + - + - + - + 

GOSS  I  24  I  136  i  20  I  120  I 

I  16.941  I  115.41  I  24  I  143.65  I 

- + - + - + - + - + 

Total  48  327  68  407 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  VOTES 


Statistic 

DF 

Value 

Prob 

Chi-Square 

3 

17.268 

0.001 

Likelihood  Ratio  Chi-Square 

3 

17.135 

0.001 

Mantel-Haenszel  Chi-Square 

1 

16.088 

0.000 

Fisher’s  Exact  Test  (2-Tail) 

6.41E-04 

Phi  Coefficient 

0.143 

Contingency  Coefficient 

0.141 

Cramer’s  V 

0.143 

Sample  Size  =  850 


Total 

550 

300 

850 
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EXPERIHEIT  2:  OVERALL  COISE1SUS 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  COISEIS 

FACILITY  COISEIS (US  REACHED) 

Frequency  I 


Expected  110 

1  YES 

1 

Total 

FAC-NAI 

1 

1 

24 

15.632 

1  526 

1  534.37 

1 

1 

550 

MAIUAL 

1 

1 

3 

11.368 

1  397 

1  388.63 

1 

1 

400 

Total 

27 

923 

950 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  COISEIS 

Statistic 

DF 

Value 

Prob 

Chi-Square 

1 

10.951 

0.001 

Likelihood  Ratio  Chi-Square 

1 

12.898 

0.000 

Continuity  Adj .  Chi-Square 

1 

9.682 

0.002 

Nantel-Haenszel  Chi-Square 

1 

10.940 

0.001 

Fisher's  Exact  Test  (Left) 

1.000 

(Right) 

4.47E-04 

(2-Tail) 

5.93E-04 

Phi  Coefficient 

0.107 

Contingency  Coefficient 

0.107 

Cramer’s  V 

0.107 

Sample  Size  =  950 


1-4 


Appendix  J.  Post-Process  Questionnaire  Results 
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Table  J.l.  Questionnaire  Responses  for  Day  1 


[-2 


Table  J.3.  Reliability  Data — Comparison  of  Participant  Responses  Between  Days. 
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Appendix  K.  Contingency  Table/ Chi-Square  Test  Results  for  Participant  Responses 
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QUESTION:  How  nWM  are  you  wMh  the  RESULTS  your  group  produoerfT 


Day  1 

OnSnal  Value  ObaarvaMona 


Treatment 

1 

2 

3 

4 

5 

8 

7 

GOSS 

6 

n 

0.00 

0.00 

0.00 

0.00 

0.00 

6.40 

Fac- Manual 

10 

5  I 

0.00 

0.00 

0.00 

0.00 

0.00 

0.60 

5.40  | 

0 

0 

0 

0 

0 

18 

9 

TEST  STATISTIC:  Chi-Square  (alpha  -  OS  A  df  -  1):  3.841 

CHt-SQ  (T  Vaiua):  0.1157407 

DECISION:  Fail  to  Rajact  Ho 


Day  2 

Ordinal  Vaiua  ObaarvaMona 


Treatment 

1 

2 

3 

4 

5 

6 

7 

GDSS 

6 

4  1 

0.00 

0.00 

0.00 

0.40 

0.40 

4.60 

4.40  | 

Fee-Manual 

1 

1 

8 

mem 

0.00 

0.00 

0.00 

0.60 

0.60 

7.20 

0  0  0  1  1 

12 

11 

25 

TEST  STATISTIC:  Chi-Square  (alpha  -  .06  8  df  -  3): 

7.815 

CHI-SQ  (T  Value):  1 .8039394 

DECISION:  Fan  to  Rajact  Ho 

Dayl 

Ordinal  Vaiua  ObaarvaMona 


Treatment 

1 

2 

3 

4 

5 

6 

7 

Fac-Manual 

10 

5 

0.00 

0.00 

0.00 

0.00 

0.00 

7.80 

7.20 

Manual 

3 

7 

0.00 

0.00 

0.00 

0.00 

0.00 

5.20 

4.60 

0 

0 

0 

0 

0 

13 

12 

TEST  STATISTIC:  Chi-Square  (alpha  -  .05  8  dl  -  1):  3.841 

CHI-SQ  (T  Value):  3.231 8378 

DECISION:  Fail  to  Reject  Ho 


Day  2 

Ordinal  Value  ObaarvaMona 


Treabnent 

1 

2 

3 

4 

5 

6 

7 

Fac-Manual 

1 

1 

8 

mom 

0.00 

0.00 

0.00 

0.60 

0.60 

5.40 

Manual 

3 

0.00 

0.00 

0.00 

0.40 

0.40 

3.60 

5.60  | 

0 

0 

0 

1 

1 

0 

14 

TEST  STATISTIC:  Chi-Square  (alpha  -  .05  &  df  -  3):  7.815 

CHI-SQ  (T  Value):  2.0833333 

DECISION:  Fan  to  Rajact  Ho 


Figure  K.l.  Contingency  Tables/Chi-Square  Test:  Results. 
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GOSS 

2 

1 
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Fac-Manual 
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13 
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TEST  STATISTIC:  Chi-Squar*  (alpha  -  .06  8  df  -  5):  11.07 

CHt-SQ  (T  Valua):  2.4572% 

DECISION:  Fail  to  Rajact  Ho 


Day  2 


Traatmant 

1 

2 

3 

4 

5 

8 

7 

QDSS 

1 

1 

2 

3 

3 

1.30 

1.20 

0.40 

1.20 

3.20 

2.40 

0.00 

Fac-Manual 

3 

2 

1 

1 

5 

3 

2  40 

1.60 

0.60 

1.60 

4.60 

3.00 

0.00 

4 

3 

1 

3 

6 

6 

0 

Traatmant 

TEST  STATISTIC.  Chi-Squara  (alpha  -  .05  &  df  -  5): 
CHI-SQ  (T  Valua):  2.2580444 

DECISION:  Fail  to  Rajaet  Ho 

Oay  1 

Ordinal  Valua  Otmarvattona 

1  2  3  4  5 

11.07 

8 

7 

Fac-Manual 

1 

1 

1 

2 

8 

2 

1.20 

3.00 

0.60 

0.00 

3.00 

6.00 

1.20 

Manual 

1 

4 

3 

2 

0.80 

2.00 

0.40 

0.00 

2.00 

4.00 

0.80 

2 

5 

1 

0 

5 

10 

2 

TEST  STATISTIC:  Chi-Squara  (alpha  -  .05  &  df 

-5): 

11.07 

CHI-SQ  (T  Valua): 

7.8188667 

DECISION: 

Fail  to  Rat  act  Ho 

Day  2 

Ordinal  Valua  Otmarvattona 

Traatmant 

1 

2 

3 

4 

5 

8 

7 

Fac-Manual 

3 

2 

1 

1 

5 

3 

2.40 

2.40 

0.60 

0.60 

4.60 

4.20 

0.00 

Manual 

1 

2 

3 

4 

1.60 

1.60 

0.40 

0.40 

3.20 

2.80 

0.00 

4 

4 

1 

1 

6 

7 

0 

TEST  STATISTIC:  Chi-Squara  (alpha  -  OS  &  df  -  5):  1 1 .07 

CHI-SQ  (T  Valua):  2.752S782 

DECISION:  Fail  to  Raj  act  Ho 


Figure  K.2.  Contingency  Tables/Chi-Squ&re  Test:  Difficulty. 
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QUESTION:  To  adwt  dagraa  area  year  froup  able  to  8TAT  fOCUtCD  on  oom pNNnf  the  lea ST 


Day  1 


Treatment 

1 

2 

3 

4 

5 

8 

7 

GOSS 

1 

1 

3 

5 

0.00 

0.80 

0.00 

0.40 

0.00 

3.20 

5.80 

Fac- Manual 

1 

5 

0 

0.00 

1.20 

0.00 

0.00 

0.00 

4.00 

0.40 

0 

2 

0 

1 

0 

8 

14 

TEST  STATISTIC:  Chl-Squara  (alpha  »  .05  8  df  -  3):  7.815 

CHI-SQ  (T  Valua):  1 .71 1 3065 

DECISION:  Fail  to  Reject  Ho 

Day2 


Treatment 

1 

2 

3 

4 

5 

8 

7 

QDSS 

3 

7 

0.00 

0.00 

0.00 

0.40 

0.00 

2.40 

7.20 

Fac-Manual 

1 

3 

11 

0.00 

0.00 

0.00 

0.00 

0.00 

3.00 

10.80 

0 

0 

0 

1 

0 

8 

16 

TEST  STATISTIC:  Chi-Square  (alpha  -  .05  A  df  -  2):  5.061 

CHI-SQ  (T  Valua):  0.6256256 

DECISION:  Fail  to  Rajact  Ho 


Day  1 

Ordinal  Valua  ObaervaSone 


Treatment 

1 

2 

3 

4 

5 

8 

7 

Fac-Manual 

1 

5 

8 

0.00 

1.20 

0.00 

0.00 

0.00 

4.20 

8.80 

Manual 

1 

2 

7 

0.00 

0.80 

0.00 

0.00 

0.00 

2.80 

6.40 

0 

2 

0 

0 

0 

7 

16 

TEST  STATISTIC:  Chl-Squara  (alpha  »  .05  &  df  -  2):  5.061 

CHI-SQ  (T  Valua):  0.5580357 

OECISION:  Fail  to  Rajact  Ho 


Day  2 

Ordinal  Valua  ObaarvaSona 


Treatment 

1 

2 

3 

4 

5 

6 

7 

Fac-Manual 

1 

3 

11 

0.00 

0.00 

0.00 

0.60 

0.00 

3.60 

10.80 

Manual 

3 

7 

0.00 

0.00 

0.00 

0.40 

0.00 

2.40 

7.20 

0 

0 

0 

1 

0 

6 

16 

TEST  STATISTIC:  Chl-Squara  (alpha  -  .05  &  df  »  2):  5.061 

CHI-SQ  (T  Valua) :  0.0250256 

OECISION:  Fan  to  Rajact  Ho 


Figure  K.3.  Contingency  Tables/Chi-Square  Test:  Stay  Focused. 
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QUESTION:  In  tmnrm  at  TIME  tFFKKNCY,  horn  prodMcNaa  area  your  group? 

Day  1 

Ordinal  Vaiua  ObaarvSana 


Treatment 

1 

2 

3 

4 

5 

8 

7 

QDSS 

1 

1 

1 

1 

2 

4 

0.40 

0.40 

0.40 

0.40 

0.40 

4.40 

3.00 

Fac- Manual 

1 

9 

5 

0.80 

0.60 

0.80 

0.80 

0.80 

8.80 

5.40 

1 

1 

1 

1 

1 

11 

9 

TEST  8TAT1ST1C:  CrvSquare  (alpha  -  .05  A  df  -  6):  1 2.502 

CHI-8  Q  (T  Vaiua).  8.0225580 

DECISION:  Fal  to  Rajact  Ho 

Day  2 

Ordinal  Vaiua  OSaarvaSona 


Treatment 

1 

2 

3 

4 

5 

8 

7 

GOSS 

4 

6 

0.00 

0.40 

0.40 

0.00 

0.00 

3.20 

6.00 

Fac-Manual 

i 

1 

4 

9 

0.00 

0.80 

0.80 

o.x 

0.00 

4.80 

9.00 

0 

1 

1 

0 

0 

8 

15 

TEST  STATISTIC:  Chi-Squara  (alpha  -  .05  &  df  -  3):  7.S15 

CHI-SO  (T  Vaiua):  1 .0888887 

DECISION:  Fall  to  Rajact  Ho 


Day  1 

Ordinal  Vaiua  OfcaarvaSna 


Treatment 

1 

2 

3 

4 

5 

6 

7 

Fac-Manual 

1 

9 

5 

0.00 

0.00 

0.60 

0.00 

0.00 

8.40 

6.00 

Manual 

5 

5 

ox 

0.00 

0.40 

0.00 

0.00 

5.60 

4.00 

0 

0 

1 

0 

0 

14 

10 

TEST  STATISTIC:  Chi-Squara  (alpha  -  .05  &  df  -  2):  5.901 

CHI-SQ  (T  Vaiua):  1.1904782 

DECISION:  Fan  to  Rajact  Ho 


Day  2 


Treatment 

1 

2 

3 

4 

5 

6 

7 

Fac-Manual 

1 

1 

4 

9 

0.00 

0.00 

0.60 

0.00 

0.00 

3.00 

10.20 

Manual 

2 

8 

000 

0.40 

0.40 

0.00 

0.00 

2.40 

6.80 

0 

1 

1 

0 

0 

6 

17 

TEST  STATISTIC:  Chi-Squara  (alpha  •  .05  4  dl  -  3):  7.815 

CHI-SO  (T  Vaiua):  1 .7973858 

DECISION:  Fail  to  Rajact  Ho 


Figure  K.4.  Contingency  Tables/Chi-Square  Test:  Time  Efficiency. 


K-5 


QUESTION:  In  him  ot  agraaa lentHdteoffeearerd.  irhol  lore!  of  COMfUCT  did  your  group 


TEST  STATISTIC:  Chi-Squara  (alpha  -  .06  A  df  -  5):  11.07 

CHI-SQ  (T  Value):  2.6488005 

DECISION:  Fail  to  Reject  Ho 


Day  2 

Ordinal  Value  Obaarvadcna 


TEST  STATISTIC:  CN-Squm  (alpha  -  .05  6  df  -  5):  1 1 .07 

CHI-SO  (T  Value):  3.0141414 

DECISION:  Fail  to  Reject  Ho 


Day  1 

Ordinal  Value  ObeervaSont 


TEST  STATISTIC:  Chi-Square  (alpha  -  .05  &  df  -  5) :  1 1 .07 

CHI-SQ  (T  Value):  3.4134615 

DECISION:  Fail  to  Reject  Ho 


Day  2 

Ordinal  Value  Observatione 


TEST  STATISTIC:  Chi-Square  (alpha  -  .05  6  df  -  4):  0.488 

CHI-SQ  (T  Value):  2.5252525 

DECISION:  Fail  to  Reject  Ho 


Figure  K.5.  Contingency  Tables/Chi- Square  Test:  Conflict. 


K-6 


QUESTION:  In  larmt  of  tBHJTY  TO  REACH  CONSENSUS,  low  preduclfra  wm  yeur  group? 

Day  1 


Treatment 

1 

2 

3 

4 

5 

6 

7 

GOSS 

1 

1 

3 

6 

0.00 

0.00 

0.00 

0.40 

0.40 

3.20 

0.00 

Fac-Manual 

5 

10 

0.00 

0.00 

0.00 

0.80 

0.00 

4.00 

9.00 

0 

0 

0 

1 

1 

8 

15 

10 

IS 

2S 


TEST  STATISTIC:  Chi-Square  (alpha  -  .OS  A  df  -  3):  7.81S 

CHI-SQ  (T  Valua):  3.28861 1 1 

DECISION:  Fan  to  Rajact  Ho 

Oay  2 


Treatment 

1 

2 

3 

4 

5 

6 

7 

GDSS 

1 

8 

0.00 

0.00 

0.00 

0.40 

0.00 

2.00 

7.80 

Fac-Manual 

1 

4 

10 

0.00 

0.00 

0.00 

o.eo 

0.00 

3.00 

11.40 

0 

0 

0 

1 

0 

s 

19 

TEST  STATISTIC:  Chi-Squara  (alpha  -  OS&dt  -  2):  5.881 

CHI-SQ  (T  Value) :  1 .8288246 

DECISION:  Fail  to  Rajact  Ho 


Oay  1 

Ordinal  Valua  ObaamaSona 


Treatment 

1 

2 

3 

4 

5 

8 

7 

Fac-Manual 

5 

10 

0.00 

0.00 

0.00 

0.00 

0.00 

4.20 

10.80 

Manual 

2 

0 

0.00 

0.00 

0.00 

0.00 

0.00 

2.80 

7.20 

0 

0 

0 

0 

0 

7 

10 

TEST  STATISTIC:  Chi-Squara  (alpha  -  OS  A  df  -  1):  3.841 

CHI-SQ  (T  Valua).  0.5281005 

DECISION:  Fail  to  Rajact  Ho 

Day  2 


Treatment 

1 

2 

3 

4 

5 

8 

7 

Fac-Manual 

1 

4 

10 

0.00 

0.00 

0.00 

0.60 

0.00 

3.00 

10.00 

Manual 

2 

8 

0.00 

0.00 

0.00 

0.40 

0.00 

2.40 

7.20 

0 

0 

0 

1 

0 

8 

18 

TEST  STATISTIC:  Chi-Squara  (Npha  -  05  &  df  -  2):  5.881 

CHI-SQ  (T  Value):  0.8258258 

DECISION:  Fail  to  Rajact  Ho 


Figure  K.6.  Contingency  Tables/Chi-Square  Test:  Consensus. 


K-7 


QUESTION:  Omni,  flaw  aatiaf fad  aru  you  arflh  Mm  OROWP  PROCESS  you  fmt  oompMad? 


Day  1 

Ordinal  Valya  ntiaarralona 


TE8T  STATISTIC:  Chi-Squara  (alpha  <■  .05  8  df  «  3):  7.81S 

CHI-SQ  (T  Vaiua):  8.8786714 

DECISION:  Fail  to  Rajact  Ho 


Days 

Ordinal  Vaiua  Obaarvadona 


TEST  STATISTIC:  Chi-Squara  (alpha  -  .06 8  df  -  4):  9.488 

CHI-SO  (T  Vaiua):  3.8888889 

DECISION:  Fail  to  Rajact  Ho 


Day  1 

Ordinal  Vaiua  ObaarvaSana 


TEST  STATISTIC:  Chi-Squara  (alpha  -  .05  &  df  -  1):  3.841 

CHI-SQ  (T  Vaiua):  0.5291005 

DECISION:  Fail  to  Rajact  Ho 


Day  2 

Ordinal  Vaiua  ObaarvaSana 


TEST  STATISTIC:  Chi-Squara  (alpha  -  .06  8  df  -  3):  7.815 

CHI-SQ  (T  Vaiua):  2.7777778 

OECISION:  Fail  to  Rajact  Ho 


Figure  K.7.  Contingency  Tables/Chi-Square  Test:  Process. 


K-8 


QUESTION:  Daaad  on  your  parcaptlona.  horn  STRUCTURED  m  tNm  group  procaaaT 

Day  1 

Ordinal  Val ua  OkMnaNM 


Traatmant 

1 

2 

3 

4 

5 

6 

7 

GOSS 

1 

3 

8 

0.00 

0.00 

0.00 

0.00 

1.00 

2.80 

5.80 

Fac-Manual 

3 

4 

0 

0.00 

0.00 

0.00 

0.00 

2.40 

4.20 

0.40 

0 

0 

0 

0 

4 

7 

14 

Traatmant 

TEST  STATISTIC:  Chi-Squwa  (alpha  -  .06  &  df  -  2). 
CHI-80  (T  Valua):  0.4464286 

DECISION:  Fan  to  Rajact  Ho 

Day  2 

OwSnal  Valua  OSaanmdona 

1  2  3  4  5 

5.901 

0 

7 

GDSS 

1 

8 

3 

0.00 

0.00 

0.00 

0.00 

0.80 

5.20 

4.00 

Fac-Manual 

1 

7 

7 

0.00 

0.00 

0.00 

0.00 

1.20 

7.80 

6.00 

0 

0 

0 

0 

2 

13 

10 

TEST  STATISTIC:  Chi-Squara  (alpha  «  .06  &  df 

-2): 

5.901 

CHI-30  (T  Valua): 

0.7051282 

OECISION: 

Fan  to  Rajact  Ho 

Dayl 

Ordinal  Valua  Obaaraatlona 

Traatmant 

1 

2 

3 

4 

5 

8 

7 

Fac-Manual 

3 

4 

8 

0.00 

0.00 

0.00 

0.00 

3.00 

5.40 

8.80 

Manual 

2 

5 

3 

0.00 

0.00 

0.00 

0.00 

2.00 

3.80 

4.40 

0 

0 

0 

0 

5 

9 

11 

TEST  STATISTIC:  Chi-Squara  (alpha  -  .05  S  df 

-2): 

5.991 

CHI-SQ  (T  Valua): 

1.6498318 

DECISION: 

Fail  to  Rajact  Ho 

Day  2 

Ordinal  Valua  Obaanradona 

Traatmant 

1 

2 

3 

4 

5 

8 

7 

Fac-Manual 

1 

7 

7 

0.00 

0.00 

0.00 

0.00 

1.20 

0.40 

5.40 

Manual 

1 

7 

2 

0.00 

0.00 

0.00 

0.00 

0.00 

5.80 

3.80 

0 

0 

0 

0 

2 

14 

9 

TEST  STATISTIC:  Chi-Squara  (alpha  -  .05  6  df  -  2):  5.901 

CHI-SQ  (T  Valua):  1 .851 551 9 

DECISION:  Fan  to  Raject  Ho 


Figure  K.8.  Contingency  Tables/Chi-Square  Test:  Structure. 


Appendix  L.  Chi-Squart/Fisher’s  Exact  Test  Results  for  Participant  Responses 


L-l 


QUESTIO*  1:  RESULTS,  TEST  1,  DAY  1 
CHI-SQUARE/FISHER  EXACT  TEST 


TABLE  OF  FACILITY  BY  LEVEL 
FACILITY  LEVEL (ORDIIAL  VALUE  OBSERVATIOI) 

Frequency  I 


Expected 

1 

61 

71 

Total 

FCMH 

1 

1 

10  1 
9.6  1 

5  I 
5.4  i 

15 

GDSS 

1 

1 

6  I 
6.4  1 

4  i 
3.6  1 

10 

Total 

16 

9 

25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

1 

0.116 

0.734 

Likelihood  Ratio  Chi-Square 

1 

0.115 

0.734 

Continuity  Adj.  Chi-Square 

1 

0.000 

1.000 

Mantel-Haenszel  Chi-Square 

1 

0.111 

0.739 

Fisher’s  Exact  Test  (Left) 

0.778 

(Right) 

0.530 

(2-Tail) 

1.000 

Phi  Coefficient 

0.068 

Contingency  Coefficient 

0.068 

Cramer's  V 

0.068 

Sample  Size  =  26 

WARNING :  25'/,  of  the  cells  have  expected  counts  less 

than  6.  Chi-Square  may  not  be  a  valid  test. 


L-2 


QUESTIOI  1:  RESULTS,  TEST  1,  DAT  2 
CHI-SQUARE/FISHER  EXACT  TEST 


FACILITY 


TABLE  OF  FACILITY  BY  LEVEL 
LEVEL (ORDIIAL  VALUE  OBSERVATIOI) 


Frequency | 
Expected  I 

41 

51 

61 

7 

FCMM  I 

1 

1  1 
0.6  I 

1  1 
0.6  I 

6  I 
7.2  i 

7 

6.6 

GDSS  | 

1 

0  I 
0.4  I 

o 

*  o 

6  I 
4.8  I 

4 

4.4 

Total 

1 

1 

12 

11 

15 


10 


25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

3 

1.894 

0.595 

Likelihood  Ratio  Chi-Square 

3 

2.594 

0.458 

Mantel-Haenszel  Chi-Square 

1 

0.190 

0.663 

Fisher's  Exact  Test  (2-Tail) 

0.888 

Phi  Coefficient 

0.275 

Contingency  Coefficient 

0.265 

Cramer's  V 

0.275 

Sample  Size  =  25 

WARNIIG:  75*/.  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


QUESTIOI  1:  RESULTS,  TEST  2,  DAY  1 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 

FACILITY  LEVEL(ORDIIAL  VALUE  OBSERVED) 

Frequency | 


Expected 

1 

61 

71 

Total 

FCMB 

1 

1 

10  I 
7.8  I 

5  1 
7.2  1 

15 

MAIL 

1 

1 

3  I 
5.2  I 

7  I 
4.8  I 

10 

Total 

13 

12 

25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

1 

3.232 

0.072 

Likelihood  Ratio  Chi-Square 

1 

3.305 

0.069 

Continuity  Adj.  Chi-Square 

1 

1.930 

0.165 

Mantel-Haenszel  Chi-Square 

1 

3.103 

0.078 

Fisher's  Exact  Test  (Left) 

0.987 

(Right) 

0.082 

(2-Tail) 

0.111 

Phi  Coefficient 

0.360 

Contingency  Coefficient 

0.338 

Cramer’s  V 

0.360 

Sample  Size  =  25 

WARMING :  25%  of  the  cells  have  expected  counts  less 

than  5.  Chi-Square  may  not  be  a  valid  test. 


L-4 


QUESTIOI  1:  RESULTS,  TEST  2,  DAY  2 
CHI-SQUARE/FISHER  EXACT  TEST 


TABLE  OF 

FACILITY  BY  LEVEL 

FACILITY 

LEVELCORDIIAL  VALUE  OBSERVED) 

Frequency  1 
Expected  I 

41 

51 

61  71 

Total 

FCMH 

1 

1 

1  1 

0.6  I 

1  1 

0.6  1 

6  1  7| 

5.4  1  8.4  I 

IS 

MARL 

1 

1 

0  1 

0.4  1 

0  i 
0.4  1 

3  1  7  1 

3.6  1  5.6  | 

10 

Total 

1 

1 

9  14 

25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 


Chi-Square 

Likelihood  Ratio  Chi-Square 
Mantel-Haenszel  Chi-Square 
Fisher’s  Exact  Test  (2-Tail) 
Phi  Coefficient 
Contingency  Coefficient 
Cramer’s  V 


DF 

Value 

Prob 

3 

2.083 

0.555 

3 

2.785 

0.426 

1 

1.910 

0.167 

0.884 

0.289 

0.277 

0.289 

Sample  Size  =  25 

WARRING:  63%  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-5 


QUEST I 01  2:  DIFFICULTY,  TEST  1,  DAY  1 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 

FACILITY  LEVEL (ORDIIAL  VALUE  OBSERVED) 


Frequency  I 
Expected  I 

11 

21 

FCMN 

■1 - 

1 

1 

1  1 
0.6  I 

1  1 

1.8  1 

GDSS 

■+**“ 

1 

1 

_  j _ - _ 

0  I 
0.4  I 

2  I 
1.2  I 

Total 

1 

3 

31 
— + 


1  I 

0.6  I 


+■ 


0  I 
0.4  I 


1 


51 


2  I 

1.8  I 


1  I 
1.2  I 


3 


61 
— + 


8  I 
7.8  I 


5  I 
5.2  I 


+ 


13 


7 |  Total 


2  I  15 
2.4  I 


2  I  10 

1.6  I 

- +  25 

4 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

5 

2.457 

0.783 

Likelihood  Ratio  Chi-Square 

5 

3.144 

0.678 

Hantel-Haenszel  Chi-Square 

1 

0.019 

0.889 

Fisher's  Exact  Test  (2-Tail) 

0.972 

Phi  Coefficient 

0.314 

Contingency  Coefficient 

0.299 

Cramer’s  V 

0.314 

Sample  Size  =  25 

WARNING:  83'/.  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-6 


QUESTIOI  2:  DIFFICULTY,  TEST  1,  DAY  2 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 


FACILITY  LEVEL(ORDIIAL  VALUE  OBSERVED) 


Frequency | 

Expected  I  1 | 

- + - +• 

FCNI  |  3  | 

I  2.4  | 

- + - +• 

GDSS  |  1  | 

I  1.6  | 

- + - +• 

Total  4 


21 

31 

2  1 
1.8  I 

1  1 
0.6  I 

.  ...  ■_  .  . _ l 

1  1 
1.2  I 

0  1 
0.4  1 

3 

1 

4|  5|  61  Total 

- + - + - + 

11  51  3  I  15 

1.8  )  4.8  I  3.6  I 

- + - + - + 

21  3|  3  I  10 

1.2  I  3.2  I  2.4  I 

- + - + - + 

3  8  6  25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

5 

2.257 

0.813 

Likelihood  Ratio  Chi-Square 

5 

2.611 

0.760 

Nantel-Haenszel  Chi-Square 

1 

0.640 

0.424 

Fisher's  Exact  Test  (2-Tail) 

0.907 

Phi  Coefficient 

0.300 

Contingency  Coefficient 

0.288 

Cramer's  V 

0.300 

Sample  Size  =  25 

WARRING :  100%  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-7 


qUESTIOI  2:  DIFFICULTY,  TEST  2,  DAY  1 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 

FACILITY  LEVEL (ORDIIAL  VALUE  OBSERVED) 


Frequency  I 

Expected  I  ll  21  3|  Si  6|  7 1  Total 

- + - + - + - + - + - + - + 

FCMH  I  1|  1|  1|  2|  8|  21  IS 

I  1.2  I  31  0.6  I  31  6|  1.2  I 

- + - + - + - + - + - + - + 

MARL  I  1|  41  0|  31  2|  0  I  10 

I  0.8  I  21  0.4  I  21  41  0.8  I 

- + - + - + - + - + - + - + 

Total  2  S  1  5  10  2  2S 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

S 

7.917 

0.161 

Likelihood  Ratio  Chi-Square 

5 

9.136 

0.104 

Nantel-Haenszel  Chi-Square 

1 

3.915 

0.048 

Fisher's  Exact  Test  (2-Tail) 

0.146 

Phi  Coefficient 

0.563 

Contingency  Coefficient 

0.490 

Cramer's  V 

0.563 

Sample  Size  =  25 

WARRING:  92%  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-8 


QUEST I 01  2:  DIFFICULTY,  TEST  2,  DAY  2 
CHI-SQUAHE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 


FACILITY  LEVEL(ORDIIAL  VALUE  OBSERVED) 


Frequency  I 
Expected  I  1 1 

- + - + 

FCMH  |  3  I 

I  2.4  I 

- + - +■ 

MARL  |  1  | 

I  1.6  I 

- + - + 

Total  4 


21  31  41  51  61  Total 

- + - + - + - + - + 

2|  1  I  1  I  51  3  I  15 

2.4  I  0.6  I  0.6  I  4.8  I  4.2  I 

- + - + - + - + - + 

21  0|  0|  3|  4  I  10 

1.6  I  0.4  I  0.4  I  3.2  I  2.8  I 

- + - + - + - + - + 

4  1  1  8  7  25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

5 

2.753 

0.738 

Likelihood  Ratio  Chi-Square 

5 

3.461 

0.629 

Mantel-Haenszel  Chi-Square 

1 

0.583 

0.445 

Fisher’s  Exact  Test  (2-Tail) 

0.926 

Phi  Coefficient 

0.332 

Contingency  Coefficient 

0.315 

Cramer’s  V 

0.332 

Sample  Size  =  25 

WARRING:  1007,  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-9 


QUEST I 01  3:  STAY  FOCUSED,  TEST  1,  DAY  1 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 

FACILITY  LEVEL (ORDIIAL  VALUE  OBSERVED) 


Frequency  I 
Expected  I 

21 

41 

61 

71 

Total 

FCHR  1 

1 

1  1 

1.2  1 

0  I 
0.6  1 

5  1 
4.8  1 

9  I 
8.4  i 

15 

GDSS  1 

1 

1  1 
0.8  I 

1  1 
0.4  I 

3  1 
3.2  I 

5  1 
5.6  I 

10 

Total 

2 

1 

8 

14 

25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 
Statistic  DF  Value  Prob 


Chi-Square 

Likelihood  Ratio  Chi-Square 
Mantel-Haenszel  Chi-Square 
Fisher’s  Exact  Test  (2-Tail) 
Phi  Coefficient 
Contingency  Coefficient 
Cramer’s  V 


3 

1.711 

0.634 

3 

2.044 

0.563 

1 

0.548 

0.459 

0.811 

0.262 

0.253 

0.262 

Sample  Size  =  25 

WARRING:  75'/,  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-10 


QUESTI01  3:  STAY  FOCUSED,  TEST  1,  DAY  2 
CHI-SQUARE/FISHER  EXACT  TEST 


TABLE  OF  FACILITY  BY  LEVEL 
FACILITY  LEVEL (ORDIIAL  VALUE  OBSERVED) 


Frequency  1 
Expected  1 

41 

61 

71 

Total 

FCMI  I 

1 

1  1 
0.6  I 

3  I 
3.6  | 

11  1 

10.8  1 

15 

GDSS  1 

1 

0  I 
0.4  I 

3  I 
2.4  I 

7  I 
7.2  I 

10 

Total 

1 

6 

18 

25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

2 

0.926 

0.629 

Likelihood  Ratio  Chi-Square 

2 

1.276 

0.528 

Mantel-Haenszel  Chi-Square 

1 

0.122 

0.726 

Fisher '8  Exact  Test  (2-Tail) 

0.799 

Phi  Coefficient 

0.192 

Contingency  Coefficient 

0.189 

Cramer's  V 

0.192 

Sample  Size  =  25 

WARNING:  67%  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-ll 


qUESTIOI  3:  STAY  FOCUSED,  TEST  2,  DAY  1 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 

FACILITY  LEVEL(ORDIIAL  VALUE  OBSERVED) 


Frequency  I 

Expected  I  2 1  6|  7 1  Total 

- + - + - + - + 

FCHH  |  1|  5|  9|  15 

I  1.2  I  4.2  I  9.6  I 

- + - + - + - + 

MARL  I  1|  2|  71  10 

I  0.8  I  2.8  I  6.4  I 

- + - + - + - + 

Total  2  7  16  25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

2 

0.558 

0.757 

Likelihood  Ratio  Chi-Square 

2 

0.572 

0.751 

Mantel-Haenszel  Chi-Square 

1 

0.004 

0.953 

Fisher’s  Exact  Test  (2-Tail) 

0.829 

Phi  Coefficient 

0.149 

Contingency  Coefficient 

0.148 

Cramer’s  V 

0.149 

Sample  Size  =  25 

UARRIRG:  67%  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-12 


QUESTION  3:  STAY  FOCUSED,  TEST  2,  DAY  2 
CHI-SQUARE/FISBER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 


FACILITY 

Frequency 


LEVEL(ORDIIAL  VALUE  OBSERVED) 


Expected 

1 

41 

61 

71 

Total 

FCNI 

1 

1 

1  1 

0.6  1 

3  1 
3.6  i 

11  1 
10.8  | 

15 

NAIL 

1 

1 

0  1 
0.4  I 

3  I 
2.4  1 

7  I 
7.2  | 

10 

Total 

1 

6 

18 

25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

2 

0.926 

0.629 

Likelihood  Ratio  Chi-Square 

2 

1.276 

0.528 

Mantel-Haenszel  Chi-Square 

1 

0.122 

0.726 

Fisher's  Exact  Test  (2-Tail) 

0.799 

Phi  Coefficient 

0.192 

Contingency  Coefficient 

0.189 

Cramer's  V 

0.192 

Sample  Size  =  25 

WARRING:  67V.  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-13 


qUESTIOI  4:  TINE  EFFICIE1CY,  TEST  1,  DAY  1 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 


FACILITY 

LEVEL (ORDIIAL  VALUE  OBSERVED) 

Frequency  I 
Expected  1 

11 

21 

3  i  41 

Total 

FCMS  1 

1 

0  I 
0.6  I 

0  I 
0.6  1 

1  1  0  1 
0.6  1  0.6  I 

15 

GDSS  1 

1 

1  1 
0.4  1 

1  1 
0.4  1 

0  1  11 
0.4  1  0.4  I 

10 

Total 

(Continued) 

1 

1 

1  1 

25 

TABLE  OF  FACILITY  BY  LEVEL 
FACILITY  LEVELCORDIIAL  VALUE  OBSERVED) 


Frequency  I 
Expected  1 

51 

6| 

71 

Total 

FCMM  1 

0  I 

9  I 

5  1 

15 

1 

0.6  I 

6.6  I 

5.4  1 

GDSS  1 

1  1 

2  1 

4  1 

10 

1 

0.4  1 

4.4  1 

3.6  1 

Total 

1 

11 

9 

25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 


Chi-Square 

Likelihood  Ratio  Chi-Square 
Mantel-Haenszel  Chi-Square 
Fisher’s  Exact  Test  (2-Tail) 
Phi  Coefficient 
Contingency  Coefficient 
Cramer’s  V 


DF 

Value 

Prob 

6 

8.923 

0.178 

6 

10.854 

0.093 

1 

2.005 

0.157 

0.097 

0.597 

0.513 

0.597 

Sample  Size  =  25 

WARHIHG:  86%  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-14 


QUESTIOI  4:  TIME  EFFIC1EICY,  TEST  1,  DAY  2 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 

FACILITY  LEVEL (ORDIIAL  VALUE  OBSERVED) 


Frequency  I 

Expected  I  21  31  6i  7 1  Total 

- + - + - + - + - + 

FCMI  I  1  I  1  I  4|  9|  15 

I  0.6  I  0.6  I  4.8  I  9| 

- + - + - + - + - + 

GDSS  I  01  01  4|  61  10 

I  0.4  I  0.4  I  3.2  I  6| 

- + - + - + - + - + 

Total  1  1  8  15  25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

3 

1.667 

0.644 

Likelihood  Ratio  Chi-Square 

3 

2.370 

0.499 

Mantel-Haenszel  Chi-Square 

1 

0.838 

0.360 

Fisher’s  Exact  Test  (2-Tail) 

0.890 

Phi  Coefficient 

0.258 

Contingency  Coefficient 

0.250 

Cramer’s  V 

0.258 

Sample  Size  =  25 

VARNIHG:  75'/.  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-15 


QUESTIOI  4:  TIME  EFFICIEICY,  TEST  2,  DAY  1 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 

FACILITY  LEVELCORDIIAL  VALUE  OBSERVED) 


Frequency  1 
Expected  I 

J- 

31 

61 

FCMI 

- 

1 

1 

1  1 
0.6  I 

9  I 
8.4  I 

MAIL 

1 

1 

0  1 
0.4  1 

5  1 
5.6  1 

Total 

T  ” 

1 

14 

71 


5  I 

6  I 
— + 
S  I 
4  I 


10 


Toted 

IS 

10 

25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

2 

1.190 

0.551 

Likelihood  Ratio  Chi-Square 

2 

1.538 

0.463 

Mantel-Haenszel  Chi-Square 

1 

1.136 

0.286 

Fisher’s  Exact  Test  (2-Tail) 

0.807 

Phi  Coefficient 

0.218 

Contingency  Coefficient 

0.213 

Cramer’s  V 

0.218 

Sample  Size  =  25 

VARIIVG:  50'/.  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  he  a  valid  test. 


L-16 


QUESTIOI  4:  TINE  EFFICIENCY,  TEST  2.  DAY  2 
CHI-SQUARE/FISHER  EXACT  TEST 


FACILITY 

Frequency 

Expected 


FCMN 


MAIL 


Total 


TABLE  OF  FACILITY  BY  LEVEL 
LEVEL (ORDINAL  VALUE  OBSERVED) 


21  3|  6|  7|  Total 

- + - + - + - + 

1  I  11  4  1  9|  15 

0.6  I  0.6  I  3.6  I  10.2  I 

- + - + - + - + 

0  I  0  1  2|  8|  10 

0.4  I  0.4  I  2.4  I  6.8  I 

- + - + - + - + 

1  1  6  17  25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

3 

1.797 

0.616 

Likelihood  Ratio  Chi-Square 

3 

2.504 

0.475 

Mantel-Haenszel  Chi-Square 

1 

1.684 

0.194 

Fisher’s  Exact  Test  (2-Tail) 

0.881 

Phi  Coefficient 

0.268 

Contingency  Coefficient 

0.259 

Cramer’s  V 

0.268 

Sample  Size  =  25 

WARRING:  75%  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-17 


qUESTIOI  S:  COVFLICT ,  TEST  1.  DAY  1 
CHI-SQUARE/FISHER  EXACT  TEST 


TABLE  OF  FACILITY  BY  LEVEL 
FACILITY  LEVELCORDIIAL  VALUE  OBSERVED) 


Frequency  I 
Expected  I 

- +. 

FCMH  I 

I 

- +. 

GDSS  | 

I 

- +• 

Total 


21  31  41 

- + - + - + 

11  3  1  II 

0.6  I  2.4  I  0.6  I 

- + - + - + 

0  1  1  I  0  1 

0.4  I  1.6  I  0.4  I 

- + - + - + 

14  1 


51 


1  I 
1.2  I 


1  I 
0.8  I 


2 


61  71  Total 

- + - + 

6|  3  I  15 

61  4.2  I 

- + - + 

41  4  I  10 

41  2.8  I 

10  7  25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

5 

2.649 

0.754 

Likelihood  Ratio  Chi-Square 

5 

3.358 

0.645 

Mantel-Haenszel  Chi-Square 

1 

1.857 

0.173 

Fisher’s  Exact  Test  (2-Tail) 

0.932 

Phi  Coefficient 

0.326 

Contingency  Coefficient 

0.310 

Cramer’s  V 

0.326 

Sample  Size  =  25 

WARRING:  92V.  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-18 


qUESTIOI  5:  COIFLICT,  TEST  1,  DAY  2 
CHI-SQUARE/FISHER  EXACT  TEST 


TABLE  OF  FACILITY  BY  LEVEL 
FACILITY  LEVEL (ORDINAL  VALUE  OBSERVED) 


Frequency 1 
Expected  I 

21 

31 

41 

51 

FCMM  1 

1 

1  1 

0.r  ' 

2  1 
2.4  I 

0  1 
0.6  I 

2  1 

1.2  1 

GDSS  I 

1 

0  I 
0.4  I 

2  1 

1.6  1 

1  1 
0.4  I 

0  1 
0.8  I 

Total 

1 

4 

1 

2 

61 


6  I 

6.6  I 


5  I 
4.4  I 


11 


T|  Total 


4  I  IS 
3.6  I 


2  I  10 
2.4  I 


6  25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

5 

3.914 

0.562 

Likelihood  Ratio  Chi-Square 

5 

5.309 

0.379 

Mantel-Haenszel  Chi-Square 

1 

0.011 

0.915 

Fisher’s  Exact  Test  (2-Tail) 

0.791 

Phi  Coefficient 

0.396 

Contingency  Coefficient 

0.368 

Cramer’s  V 

0.396 

Sample  Size  =  25 

WARRING:  92'/.  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-19 


QUESTIOI  5:  COIFLICT,  TEST  2.  DAY  1 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 

FACILITY  LEVEL (ORDIIAL  VALUE  OBSERVED) 


Frequency  I 

Expected  I  2 1  3 1 

- + - + - + 

FCNN  I  II  31 

I  0.6  I  2.4  I 
- + - + - + 


41 

51 

61 

7 1  Total 

1  1 

1  i 

6  I 

3  1  15 

0.6  1 

0.6  1 

7.8  1 

3  1 

» _ 

MANL 


0|  1  |  01  01  71  21  10 

0.4  I  1.6  I  0.4  I  0.4  I  5.2  I  2  I 


■+ - + - + - + - + - + - + 


Total  1  4  1  1  13  5  25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

5 

3.413 

0.637 

Likelihood  Ratio  Chi-Square 

5 

4.477 

0.483 

Hantel-Haenszel  Chi-Square 

1 

1.563 

0.211 

Fisher's  Exact  Test  (2-Tail) 

0.874 

Phi  Coefficient 

0.370 

Contingency  Coefficient 

0.347 

Cramer’s  V 

0.370 

Sample  Size  =  25 

WARNING:  83'/,  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-20 


QUESTION  6:  CONFLICT,  TEST  2,  DAY  2 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 

FACILITY 

LEVELCORDINAL  VALUE  OBSERVED) 

Frequency  1 
Expected  1 

21  31 

51  61 

71 

Total 

FCMN  1 

1 

1  1  2  1 
0.6  I  1.8  I 

2  1  6  1 

1.2  I  6.6  I 

4  1 
4.8  I 

15 

MANL  1 

1 

0  1  11 
0.4  I  1.2  I 

0  1  5  1 

0.8  I  4.4  1 

4  1 
3.2  1 

10 

Total 

1  3 

2  11 

8 

25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

4 

2.525 

0.640 

Likelihood  Ratio  Chi-Square 

4 

3.583 

0.465 

Mantel-Haenszel  Chi-Square 

1 

1.132 

0.287 

Fisher’s  Exact  Test  (2-Tail) 

0.842 

Phi  Coefficient 

0.318 

Contingency  Coefficient 

0.303 

Cramer’s  V 

0.318 

Sample  Size  =  25 

WARNING:  90/  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-21 


QUESTION  6:  REACH  CONSENSUS ,  TEST  1, 
CHI-SQUARE/FISHER  EXACT  TEST 

DAY  1 

TABLE  OF  FACILITY  BY  LEVEL 

FACILITY 

LEVEL(ORDINAL  VALUE  OBSERVED) 

Frequency  I 
Expected  I 

41  51 

61 

7 1  Total 

FCNN  I 

1 

0  1  0  1 
0.6  I  0.6  I 

5  I 

4.8  I 

10  I  15 

9  1 

GOSS  I 

1 

11  11 
0.4  I  0.4  1 

3  1 

3.2  1 

5  1  10 

6  I 

Total 

1  1 

8 

15  25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

3 

3.299 

0.348 

Likelihood  Ratio  Chi-Square 

3 

3.970 

0.265 

Mantel-Haenszel  Chi-Square 

1 

2.202 

0.138 

Fisher’s  Exact  Test  (2-Tail) 

0.317 

Phi  Coefficient 

0.363 

Contingency  Coefficient 

0.341 

Cramer’s  V 

0.363 

Sample  Size  =  25 

WARNING:  75'/,  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-22 


QUESTION  6:  REACH  COISEISUS,  TEST  1,  DAY  2 
CHI-SQUARE/FISHER  EXACT  TEST 


TABLE  OF  FACILITY  BY  LEVEL 


FACILITY 

Frequency  I 
Expected  I 

- + 

FCMH  I 
I 

- +■ 

GDSS  I 
I 

- +• 

Total 


LEVEL (ORDIIAL  VALUE  OBSERVED) 

41  6|  7|  Total 

- + - + - + 

11  41  10  |  16 

0.6  |  31  11.4  I 

- + - + - + 

0|  11  9|  10 

0.4  |  2|  7.6  | 

- + - + - + 

1  6  19  26 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

2 

1.930 

0.381 

Likelihood  Ratio  Chi-Square 

2 

2.360 

0.307 

Mantel-Haenszel  Chi-Square 

1 

1.692 

0.193 

Fisher’s  Exact  Test  (2-Tail) 

0.460 

Phi  Coefficient 

0.278 

Contingency  Coefficient 

0.268 

Cramer’s  V 

0.278 

Sample  Size  =  25 

WARNING:  67'/,  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-23 


QUEST I 01  6:  REACH  COISEISUS,  TEST  2,  DAT  1 
CHI-SQUARE/FISHER  EXACT  TEST 


TABLE  OF  FACILITY  BY  LEVEL 


FACILITY 

LEVEL (ORDIIAL  VALUE  OBSERVED) 

Frequency  I 
Expected  1 

6i 

71 

Total 

FCHV 

1 

1 

5  t 
4.2  1 

10  I 
10.8  I 

15 

MAIL 

1 

1 

2  1 
2.8  I 

8  i 
7.2  1 

10 

Total 

7 

18 

25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

1 

0.529 

0.467 

Likelihood  Ratio  Chi-Square 

1 

0.544 

0.461 

Continuity  Adj .  Chi-Square 

1 

0.074 

0.785 

Mantel-Haenszel  Chi-Square 

1 

0.508 

0.476 

Fisher's  Exact  Test  (Left) 

0.882 

(Right) 

0.399 

(2-Tail) 

0.659 

Phi  Coefficient 

0.145 

Contingency  Coefficient 

0.144 

Cramer’s  V 

0.145 

Sample  Size  =  25 

VARIIMG:  50%  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-24 


QUEST I 01  6:  REACH  COISEISUS,  TEST  2,  DAY  2 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 


FACILITY 

LEVEL (ORDINAL  VALUE  OBSERVED) 

Frequency  I 
Expected  I 

41 

61 

71 

Total 

FCMN  I 

1 

1  1 
0.6  I 

4  1 

3.6  I 

10  I 
10.8  I 

15 

MANL  I 

1 

0  1 
0.4  I 

2  I 
2.4  I 

8  I 
7.2  I 

10 

Total 

1 

6 

18 

25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

2 

0.926 

0.629 

Likelihood  Ratio  Chi-Square 

2 

1.282 

0.527 

Mantel-Haenszel  Chi-Square 

1 

0.871 

0.351 

Fisher’ s  Exact  Test  (2-Tail) 

1.000 

Phi  Coefficient 

0.192 

Contingency  Coefficient 

0.189 

Cramer’s  V 

0.192 

Sample  Size  =  25 

WARNING:  67%  of  the  cells  have  expected  counts  less 
than  6.  Chi-Square  may  not  be  a  valid  test. 


L-25 


QUEST I 01  7:  GROUP  PROCESS,  TEST  1,  DAY  1 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 


FACILITY  LEVEL(ORDIIAL  VALUE  OBSERVED) 


Frequency  I 
Expected  I  1| 

- + - +• 

FCHH  I  0  I 

I  0.6  I 

- + - +■ 

GDSS  I  1  I 

I  0.4  I 

- + - +. 

Total  1 


61  71 

- + - + 

5  1  10  I 

61  8.4  I 

- + - + 

5  I  4| 

41  5.6  I 

10  14 


Total 

15 

10 

25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

2 

2.679 

0.262 

Likelihood  Ratio  Chi-Square 

2 

3.036 

0.219 

Mantel-Haenszel  Chi-Square 

1 

2.367 

0.124 

Fisher’s  Exact  Test  (2-Tail) 

0.288 

Phi  Coefficient 

0.327 

Contingency  Coefficient 

0.311 

Cramer’s  V 

0.327 

Sample  Size  =  25 

WARRING:  50'/.  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-26 


QUESTIOS  7:  GROUP  PROCESS,  TEST  1,  DAY  2 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 

FACILITY  LEVEL(ORDIHAL  VALUE  OBSERVED) 


Frequency  I 

Expected  I  2|  4|  Si  6|  7|  Total 

- + - + - + - + - + - + 

FCMH  I  21  11  01  41  81  15 

I  1.2  I  0.6  I  0.6  I  3.6  I  91 

- + - + - + - + - + - + 

GDSS  |  01  01  11  21  71  10 

I  0.8  I  0.4  I  0.4  I  2.4  I  6| 

- + - + - + - + - + - + 

Total  2  1  1  6  IS  25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

4 

3.889 

0.421 

Likelihood  Ratio  Chi-Square 

4 

5.285 

0.259 

Mantel-Haenszel  Chi-Square 

1 

1.508 

0.219 

Fisher’s  Exact  Test  (2-Tail) 

0.614 

Phi  Coefficient 

0.394 

Contingency  Coefficient 

0.367 

Cramer’s  V 

0.394 

Sample  Size  =  25 

WARNING:  807,  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 


L-27 


qUESTIOV  7:  GROUP  PROCESS,  TEST  2,  DAY  1 
CHI-SQUARE/FISHER  EXACT  TEST 


TABLE  OF  FACILITY  BY  LEVEL 
FACILITY  LEVEL (ORDIIAL  VALUE  OBSERVED) 


Frequency  1 
Expected  1 

61 

71 

Total 

FCHN  I 

1 

S  I 
4.2  | 

10  1 

10.8  1 

15 

MARL  I 

1 

2  I 
2.8  | 

8  1 
7.2  1 

10 

Total 

7 

18 

25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

1 

0.529 

0.467 

Likelihood  Ratio  Chi-Square 

1 

0.544 

0.461 

Continuity  Adj.  Chi-Square 

1 

0.074 

0.785 

Mantel-Haenszel  Chi-Square 

1 

0.508 

0.476 

Fisher’s  Exact  Test  (Left) 

0.882 

(Right) 

0.399 

(2-Tail) 

0.659 

Phi  Coefficient 

0.145 

Contingency  Coefficient 

0.144 

Cramer's  V 

0.145 

Sample  Size  =  25 

WARNING:  50%  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 
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QUESTIOI  7:  GROUP  PROCESS.  TEST  2,  DAY  2 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 


FACILITY  LEVEL (ORDI1AL  VALUE  OBSERVED) 


Frequency  I 
Expected  I 

21 

41 

61 

7 

FCMN  I 

2  I 

1  1 

4  I 

8 

1 

1.2  I 

0.6  I 

3.6  I 

9.6 

NAIL  I 

0  I 

0  I 

2  I 

8 

1 

0.8  I 

0.4  i 

2.4  1 

6.4 

Total  2  1  6  16 


Total 

16 

10 

25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

3 

2.778 

0.427 

Likelihood  Ratio  Chi-Square 

3 

3.832 

0.280 

Mantel-Haenszel  Chi-Square 

1 

2.481 

0.115 

Fisher’s  Exact  Test  (2-Tail) 

0.654 

Phi  Coefficient 

0.333 

Contingency  Coefficient 

0.316 

Cramer '8  V 

0.333 

Sample  Size  =  25 

WARRING:  75%  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 
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QUESTIOI  8:  STEUCTUEE,  TEST  1,  DAY  1 
CHI-SQUAEE/FISHEE  EXACT  TEST 


TABLE  OF  FACILITY  BY  LEVEL 


FACILITY 

Frequency  I 
Expected  | 

- + 

FCNX  I 

I 

- + 

GDSS  I 

I 

- + 

Total 


LEVELCOEDIKAL  VALUE  OBSEEVED) 

51  6|  71  Total 

- + - + - + 

3|  41  8|  15 

2.4  I  4.2  |  8.4  I 

- + - — + - + 

11  31  6|  10 

1.6  I  2.8  I  5.6  I 

- + - + - + 

4  7  14  25 


STATISTICS  FOE  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

2 

0.446 

0.800 

Likelihood  Eatio  Chi-Square 

2 

0.470 

0.791 

Mantel-Haenszel  Chi-Square 

1 

0.286 

0.593 

Fisher’s  Exact  Test  (2-Tail) 

1.000 

Phi  Coefficient 

0.134 

Contingency  Coefficient 

0.132 

Cramer’s  V 

0.134 

Sample  Size  =  25 

VAENIHG:  67%  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 
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QUEST I OK  8:  STRUCTURE,  TEST  1,  DAY  2 
CHI-SQUARE/FISHER  EXACT  TEST 

TABLE  OF  FACILITY  BY  LEVEL 

FACILITY  LEVEL(ORDIIAL  VALUE  OBSERVED) 


Frequency I 

Expected  I  5 1  6 1 

- + - + - +• 

FCMI  I  1  I  71 

I  1.2  I  7.8  I 

- + - + - +• 

GDSS  I  1  I  61 

I  0.8  I  5.2  I 

- + - + - +. 


71 


7  I 
6  I 


3  I 

4  I 


Total 

15 

10 


Total  2  13  10  25 


STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

2 

0.705 

0.703 

Likelihood  Ratio  Chi-Square 

2 

0.716 

0.699 

Mantel-Haenszel  Chi-Square 

1 

0.610 

0.435 

Fisher's  Exact  Test  (2-Tail) 

0.835 

Phi  Coefficient 

0.168 

Contingency  Coefficient 

0.166 

Cramer’s  V 

0.168 

Sample  Size  =  25 

WARNING:  50*/,  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 
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QUESTIOI  8:  STRUCTURE.  TEST  2.  DAY  1 
CHI-SQUARE/FISHER  EXACT  TEST 


TABLE  OF  FACILITY  BY  LEVEL 

LEVEL(ORDIIAL  VALUE  OBSERVED) 


FACILITY 

Frequency I 
Expected  I 
- +. 

FCNN  I 

I 

- +. 

MAUL  I 

I 

- + 

Total 


51  61 

3  1  4  1 

31  5.4  I 

2  1  5  1 

2|  3.6  I 

— + - +. 

5  9 


71 

Total 

- + 

8  I 

15 

6.6  I 

3  I 

10 

4.4  I 
- + 

11 

25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

2 

1.650 

0.438 

Likelihood  Ratio  Chi-Square 

2 

1.664 

0.435 

Mantel-Haenszel  Chi-Square 

1 

0.538 

0.463 

Fisher’s  Exact  Test  (2-Tail) 

0.580 

Phi  Coefficient 

0.2S7 

Contingency  Coefficient 

0.249 

Cramer’s  V 

0.257 

Sample  Size  =  25 

WARRING:  €7%  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 
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qUESTIOI  8:  STRUCTURE,  TEST  2,  DAY  2 
CHI-SQUARE/FISHER  EXACT  TEST 


TABLE  OF  FACILITY  BY  LEVEL 


FACILITY 

LEVEL (ORDIIAL  VALUE  OBSERVED) 

Frequency | 
Expected  I 

51 

6| 

71 

Total 

FCMN 

1 

1 

1  1 

1.2  1 

7  1 
8.4  I 

7  I 
5.4  I 

15 

MARL 

1 

1 

1  1 
0.8  | 

7  1 
5.6  | 

2  I 
3.6  I 

10 

Total 

2 

14 

9 

25 

STATISTICS  FOR  TABLE  OF  FACILITY  BY  LEVEL 


Statistic 

DF 

Value 

Prob 

Chi-Square 

2 

1.852 

0.396 

Likelihood  Ratio  Chi-Square 

2 

1.935 

0.380 

Mantel-Haenszel  Chi-Square 

1 

1.434 

0.231 

Fisher’s  Exact  Test  (2-Tail) 

0.333 

Phi  Coefficient 

0.272 

Contingency  Coefficient 

0.263 

Cramer’s  V 

0.272 

Sample  Size  =  25 

WARNING:  50*/,  of  the  cells  have  expected  counts  less 
than  5.  Chi-Square  may  not  be  a  valid  test. 
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Appendix  M.  Sign  Test  Results  for  Participant  Responses  and  Questionnaire 

Reliability 


M-l 


Table  M.l.  Sign  Test  for  Experiment  1,  Results. 


PERSON 

QDSS 

FAC-MAN 

OBJECTIVE 

COMPARISON 

SION  TEST 

SUBJECTIVE 

COMPARISON 

SION  TEST 

RELIABILITY 

SIGN  TEST 

1-1 

« 

7 

Lower 

- 

Lower 

- 

+ 

1-2 

6 

6 

Equal 

0 

Lower 

B 

w 

1-3 

7 

7 

Equal 

0 

Lower 

■ 

1-4 

6 

6 

Equal 

0 

Lower 

. 

- 

1-3 

6 

7 

Lower 

- 

Equal 

0 

- 

2-1 

6 

6 

Equal 

0 

Equal 

0 

+ 

2-2 

6 

T 

Lower 

. 

Equal 

0 

- 

2-3 

6 

6 

Equal 

0 

Lower 

2-4 

6 

6 

Equal 

0 

Equal 

0 

+ 

2-5 

7 

6 

Higher 

+ 

Higher 

+ 

+ 

■ 

non 

irnmni 

4 

n : 

4 

6 

10 

P(x  <  T): 

0.3123 

0.1094 

0.377 

P(x  >  T): 

0.6375 

0.9906 

0.623 

T  needed  to  Reject  Hoi 

4 

3 

7 

■■■■■■ 

■■■■■■ 

Table  M.2.  Sign  Test  for  Experiment  2,  Results. 


PERSON 

FAC-MAN 

MANUAL 

OBJECTIVE 

COMPARISON 

SIGN  TEST 

SUBJECTIVE 

COMPARISON 

SION  TEST 

RELIABILITY 

SIGN  TEST 

5-1 

7 

7 

Equal 

0 

Equal 

0 

+ 

5-2 

6 

6 

Equal 

0 

Equal 

0 

+ 

5-3 

6 

6 

Equal 

0 

Equal 

0 

+ 

5-4 

6 

6 

Equal 

0 

Equal 

0 

+ 

5-5 

6 

7 

Lower 

- 

Equal 

0 

- 

6-1 

6 

6 

Equal 

0 

Lower 

- 

- 

6-2 

6 

7 

Lower 

- 

Lower 

- 

+ 

6-3 

5 

6 

Lower 

- 

Lower 

- 

+ 

6-4 

6 

6 

Equal 

0 

Equal 

0 

+ 

6-5 

4 

7 

Lower 

- 

Lower 

- 

+ 

0 

8 

n: 

4 

4 

10 

P(x  <  T): 

0.0625 

0.0625 

0.9893 

P(r  >  T): 

0.9375 

0  9375 

0.0107 

bsbobhhi 

■  ■ 

4 

7 

[Decision  (alphas. 03  compare! 

.10  reliability)! 

_ 

Reject  Ho 

QUESTION:  How  satisfied  are  you  with  the  RESULTS  your  group  produced? 
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Table  M.3.  Sign  Test  for  Experiment  1,  Difficulty. 


Table  M.4.  Sign  Test  for  Experiment  2,  Difficulty. 


OBJECTIVE 


Daclalon  (alphas. 05  comparti  .10  reliability)! 


SUBJECTIVE 

COMPARISON 

Equal 

Equal _ 

_ Lower _ 

Equal 

Equal 

Equal 

Higher 

_ Equal _ 

_ Equal _ 

Equal 


QUESTION:  Taking  all  things  into  consideration,  how  DIFFICULT  was  this  group 

process? 


M-3 


Table  M.5.  Sign  Test  for  Experiment  1,  Stay  Focused. 


PERSON 

QDSS 

PAC-MAN 

OBJECTIVE 

COMPARISON 

SION  TEST 

SUBJECTIVE 

COMPARISON 

SION  TEST 

RELIABILITY 

SION  TEST 

l-l 

7 

7 

Equal 

0 

Equal 

0 

+ 

1.3 

2 

7 

Lower 

- 

Equal 

0 

. 

1-3 

6 

7 

Lower 

- 

Lower 

- 

+ 

1-4 

4 

7 

Lower 

- 

Equal 

0 

- 

1-5 

6 

7 

Lower 

- 

Equal 

0 

_ 

2-1 

6 

6 

Equal 

0 

Higher 

+ 

- 

2-3 

6 

7 

Lower 

- 

Equal 

0 

2-3 

7 

7 

Equal 

0 

Equal 

0 

+ 

2-4 

6 

7 

Lower 

- 

Equal 

0 

- 

2-5 

7 

7 

Equal 

0 

Higher 

+ 

- 

T: 

0 

2 

3 

n: 

6 

3 

10 

P(x  <  T): 

0.0156 

0.675 

0.1719 

P(X  >  T)- 

0.9644 

0.125 

0.6261 

T  needed  to  Reject  Hoi 

5 

3 

7 

Decision  (alpha  =  .OS  compare*  .10  reliability)! 

Table  M.6.  Sign  Test  for  Experiment  2,  Stay  Focused. 


PERSON 

FAC-MAN 

MANUAL 

OBJECTIVE 

COMPARISON 

SION  TEST 

SUBJECTIVE 

COMPARISON 

SION  TEST 

RELIABILITY 

SION  TEST 

5-1 

7 

7 

Equal 

0 

Equal 

0 

+ 

5-2 

7 

6 

Higher 

+ 

Lower 

- 

. 

3-3 

7 

7 

Equal 

0 

Equal 

0 

+ 

5-4 

7 

6 

Higher 

+ 

Equal 

0 

- 

5-5 

6 

6 

Equal 

0 

Equal 

0 

+ 

6-1 

7 

7 

Equal 

0 

Lower 

- 

- 

6-3 

6 

7 

Lower 

- 

Equal 

0 

- 

6-3 

7 

7 

Equal 

0 

Equal 

0 

+ 

6-4 

6 

7 

Lower 

- 

Lower 

- 

+ 

6-5 

4 

6 

Lower 

- 

Lower 

- 

+ 

T. 

2 

0 

6 

n 

5 

4 

10 

P(x  <  T). 

0.5 

0.0625 

0.6261 

P(x  >  T) 

0.5 

0.9375 

0.1719 

T  needed  to  Reject  Hoi 

4 

4 

7 

Decision  (alphas. 05  compare*  .10  reliability)! 

QUESTION:  To  what  degree  was  your  group  able  to  STAY  FOCUSED  on 
completing  the  task? 
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Table  M.7.  Sign  Test  for  Experiment  1,  Time  Efficiency. 


PERSON 

ODSS 

FAC- MAN 

OBJECTIVE 

COMPARISON 

SIGN  TEST 

SUBJECTIVE 

COMPARISON 

SIGN  TEST 

RELIABILITY 

SIGN  TEST 

1-1 

3 

7 

Lower 

- 

Lower 

- 

+ 

1-3 

! 

7 

Lower 

- 

Lower 

. 

+ 

1-3 

3 

7 

Lower 

- 

Lower 

- 

+ 

1-4 

4 

7 

Lower 

- 

Lower 

- 

+ 

1-5 

7 

7 

Equal 

0 

Lower 

- 

- 

3-1 

6 

6 

Equal 

0 

Equal 

0 

+  f 

3-3 

6 

7 

Lower 

- 

Equal 

0 

- 

3-3 

6 

7 

Lower 

- 

Lower 

- 

+ 

3-4 

6 

6 

Equal 

0 

Equal 

0 

+ 

3-3 

T 

7 

Equal 

0 

Higher 

+ 

- 

T: 

0 

1 

7 

n: 

6 

7 

10 

P(*  <  T): 

0.0136 

0.0633 

0.9433 

P(r  >  T): 

0.9644 

0.9373 

0.0347 

T  needed  to  Reject  Hoi 

3 

6 

7 

|Oecielon  (elph 

»=.OS  comperei  ,10  reliability)* 

Reject  Ho 

Table  M.8.  Sign  Test  for  Experiment  2,  Time  Efficiency. 


PERSON 

FAC-MAN 

MANUAL 

OBJECTIVE 

COMPARISON 

SIGN  TEST 

SUBJECTIVE 

COMPARISON 

SIGN  TEST 

RELIABILITY 

SIGN  TEST 

3-1 

7 

7 

Equal 

0 

Equal 

0 

+ 

3-3 

6 

6 

Equal 

0 

Lower 

- 

- 

5-3 

6 

7 

Lower 

- 

Lower 

- 

+ 

3-4 

6 

6 

Equal 

0 

Equal 

0 

+ 

5-3 

7 

7 

Equal 

0 

Lower 

- 

- 

6-1 

3 

7 

Lower 

. 

Lower 

- 

+ 

6-3 

3 

6 

Lower 

- 

Lower 

- 

+ 

6-3 

6 

6 

Equal 

0 

Lower 

- 

- 

6-4 

6 

6 

Equal 

0 

Equal 

0 

■f 

6-3 

6 

6 

Equal 

0 

Equal 

0 

+ 

T 

0 

0 

7 

n: 

3 

6 

10 

P(r  <  T): 

0.133 

0.0156 

0.9453 

P(r  >  T): 

0.673 

0.9844 

0-0547 

T  needed  to  Reject  Ho* 

3 

3 

7 

|Deciaion  (alpb 

a  =  .0ft  compare! 

.10  reliability)* 

Reject  Ho 

QUESTION:  In  terms  of  TIME  EFFICIENCY,  how  productive  was  your  group? 
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Table  M.9.  Sign  Test  for  Experiment  1,  Conflict. 


PERSON 

GDSS 

FAC-MAN 

OBJECTIVE 

COMPARISON 

SIGN  TEST 

SUBJECTIVE 

COMPARISON 

SIGN  TEST 

RELIABILITY 

SIGN  TEST 

1-1 

6 

3 

Higher 

+ 

Equal 

0 

- 

1-2 

ft 

6 

Equal 

0 

Equal 

0 

+ 

1-3 

5 

ft 

Lower 

- 

Higher 

+ 

1-4 

3 

2 

Higher 

+ 

Higher 

+ 

+ 

1-3 

7 

7 

Equal 

0 

Equal 

0 

+ 

2-1 

4 

5 

Lower 

. 

Equal 

0 

3-2 

7 

7 

Equal 

0 

Equal 

0 

+ 

3-3 

3 

2 

Higher 

+ 

Equal 

0 

- 

3-4 

3 

4 

Lower 

- 

Equal 

0 

. 

2-5 

ft 

6 

Equal 

0 

Lower 

bhh 

■  ■ 

mmmm 

■tfaasxfliC 

4 

n: 

6 

3 

10 

P{x  <  T ): 

0.6563 

0.873 

0.377 

P(*  >  T): 

0.3437 

0123 

0.623 

5 

3 

7 

[Decision  (alpha  =  .03  compare:  .10  reliability): 

Table  M.10.  Sign  Test  for  Experiment  2,  Conflict. 


PERSON 

FAC-MAN 

MANUAL 

OBJECTIVE 

COMPARISON 

SIGN  TEST 

SUBJECTIVE 

COMPARISON 

SIGN  TEST 

RELIABILITY 

SIGN  TEST 

3-1 

ft 

6 

Equal 

0 

Equal 

0 

+ 

3-2 

6 

7 

Lower 

- 

Equal 

0 

5-3 

7 

7 

Equal 

0 

Higher 

+ 

3-4 

6 

6 

Equal 

0 

Equal 

0 

+ 

3-5 

3 

6 

Lower 

- 

4'  Equal 

o 

6-1 

6 

7 

Lower 

- 

Higher 

+ 

6-2 

5 

6 

Lower 

- 

Higher 

+ 

6-3 

6 

6 

Equal 

0 

Higher 

+ 

6-4 

6 

6 

Equal 

0 

Higher 

+ 

6-3 

5 

6 

Lower 

- 

Higher 

+ 

T: 

0 

6 

2 

n : 

5 

6 

10 

P(X  <  T): 

0.0313 

1 

0.9047 

P(T  >  T> 

0.9687 

0 

0  9433 

T  needed  to  Reject  Hoi 

« 

5 

7 

Decieion  (alphas. Oft  compare:  .10  reliability)! 

Reject  Ho 

QUESTION:  In  terms  of  agreement /disagreement,  what  level  of  CONFLICT  did 
your  group  experience? 
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Table  M.ll.  Sign  Test  for  Experiment  1,  Consensus. 


PERSON 

QDSS 

PAC-MAN 

OBJECTIVE 

COMPARISON 

SION  TEST 

SUBJECTIVE 

COMPARISON 

SION  TEST 

RELIABILITY 

SION  TEST 

1-1 

7 

7 

Equal 

0 

Lower 

. 

. 

1-2 

6 

7 

Lower 

. 

Equal 

0 

. 

1-3 

5 

7 

Lower 

. 

Lower 

. 

+ 

1-4 

4 

7 

Lower 

. 

Lower 

. 

+ 

1-3 

7 

7 

Equal 

0 

Equal 

0 

+ 

2-1 

6 

6 

Equal 

0 

Equal 

0 

+ 

2-2 

7 

7 

Equal 

0 

Equal 

0 

+ 

2-3 

7 

6 

Higher 

+ 

Equal 

0 

. 

2-4 

7 

7 

Equal 

0 

Equal 

0 

+ 

2-3 

7 

7 

Equal 

0 

Higher 

+ 

- 

HUUH 

6 

n: 

4 

4 

10 

P(X  <  T): 

0.3123 

0.3123 

0.6281 

P{T  >  T): 

0.6673 

0.6673 

0.1719 

T  need ed  to  Reject  Hoi 

4 

4 

7 

■■1 

■■■■ 

Table  M.12.  Sign  Test  for  Experiment  2,  Consensus. 


PERSON 

FAC-MAN 

MANUAL 

OBJECTIVE 

COMPARISON 

SIGN  TEST 

SUBJECTIVE 

COMPARISON 

SION  TEST 

RELIABILITY 

SION  TEST 

3-1 

7 

7 

Equal 

0 

Equal 

0 

+ 

3-2 

7 

6 

Higher 

+ 

Equal 

0 

. 

3-3 

7 

7 

Equal 

0 

Lower 

. 

. 

3-4 

6 

6 

Equal 

0 

Equal 

0 

+ 

5-5 

6 

7 

Lower 

- 

Lower 

. 

+ 

6-1 

6 

7 

Lower 

- 

Lower 

- 

+ 

6-2 

6 

7 

Lower 

- 

Lower 

- 

+ 

6-3 

6 

6 

Equal 

0 

Lower 

. 

. 

6-4 

6 

7 

Lower 

- 

Equal 

0 

. 

6-5 

4 

6 

Lower 

. 

Lower 

- 

+ 

mm 

0 

6  1 

■BH 

6 

10 

P(r  <  T): 

0.1094 

0.0156 

0.8261 

P{r  >  T). 

0.6906 

0.9644 

0.1719 

mDHi 

5 

7 

[Decision  (alphas. 05  compare!  -10  reliability)! 

QUESTION:  In  terms  of  ABILITY  TO  REACH  CONSENSUS,  how  productive  was 
your  group? 
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Table  M.13.  Sign  Test  for  Experiment  1,  Process. 


PERSON 

ODSS 

FAC.  MAN 

OBJECTIVE 

COMPARISON 

SION  TEST 

SUBJECTIVE 

COMPARISON 

SION  TEST 

RELIABILITY 

SION  TEST 

1-1 

6 

7 

Lower 

- 

Lower 

. 

+ 

1-3 

1 

6 

Lower 

- 

Lower 

. 

+ 

1-3 

6 

7 

Lower 

- 

Lower 

_ 

+ 

1.4 

6 

6 

Equal 

0 

Equal 

0 

+ 

1.5 

6 

7 

Lower 

- 

Equal 

0 

. 

3-1 

5 

6 

Lower 

- 

Equal 

0 

- 

3-3 

7 

7 

Equal 

0 

Equal 

0 

+ 

3-3 

6 

6 

Equal 

0 

Equal 

0 

+ 

3-4 

6 

6 

Equal 

0 

Equal 

0 

+ 

3-3 

7 

7 

Equal 

0 

Equal 

0 

+ 

T: 

0 

0 

8 

n: 

3 

3 

10 

P(x  <  T): 

0.0313 

0.133 

0.9893 

P(x  >  T); 

0  9637 

0.875 

0.0107 

T  needed  to  Reject  Hoi 

* 

3 

7 

Decision  (alphas. 05  compare)  .10  reliability )i 

Reject  Ho 

Table  M.14.  Sign  Test  for  Experiment  2,  Process. 


PERSON 

FAC-MAN 

MANUAL 

OBJECTIVE 

COMPARISON 

SIGN  TEST 

SUBJECTIVE 

COMPARISON 

SION  TEST 

RELIABILITY 

SIGN  TEST 

3-1 

7 

7 

Equal 

0 

Equal 

0 

+ 

5-2 

7 

7 

Equal 

0 

Equal 

0 

+ 

5-3 

7 

6 

Higher 

+ 

Higher 

+ 

+ 

5-4 

6 

6 

Equal 

0 

Equal 

0 

+ 

5-5 

7 

7 

Equal 

0 

Lower 

- 

- 

6-1 

3 

7 

Lower 

- 

Lower 

. 

+ 

6-3 

2 

7 

Lower 

- 

Lower 

- 

+ 

6-3 

6 

7 

Lower 

- 

Equal 

0 

- 

6-4 

7 

7 

Equal 

0 

Equal 

0 

+ 

6-3 

4 

6 

Lower 

- 

Lower 

- 

+ 

1 _ 1 _ 

1 

1 

8 

n: 

5 

5 

10  j 

P(x  <  T): 

0.1875 

0.1875 

0.9893 

P(X  >  7*): 

0.8125 

0.8125 

0.0107 

T  needed  to  Reject  Hoi 

4 

4 

7 

[Decision  (alph 

ass. 05  compare) 

.10  reliability)! 

Reject  Ho 

QUESTION:  Overall,  how  satisfied  are  you  with  the  GROUP  PROCESS  you  just 
completed? 
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Table  M.15.  Sign  Test  for  Experiment  1,  Structure. 


OBJECTIVE 

SUBJECTIVE 

RELIABILITY 

PERSON 

QD8S 

PAC-M  AN 

COMPARISON 

SION  TEST 

COMPARISON 

SION  TEST 

SION  TEST 

1*1 

7 

6 

Higher 

+ 

Lower 

- 

- 

1-2 

7 

7 

Equal 

0 

Lower 

- 

- 

1-3 

6 

3 

Higher 

+ 

Higher 

+ 

+ 

1-4 

7 

ft 

Highet 

+ 

Equal 

0 

- 

1-3 

7 

7 

Equal 

0 

Equal 

0 

+ 

2-1 

6 

5 

Higher 

+ 

Equal 

0 

- 

2-2 

6 

7 

Lower 

. 

Equal 

0 

- 

2-3 

6 

7 

Lower 

- 

Equal 

0 

- 

2-4 

7 

7 

Equal 

0 

Equal 

0 

+ 

2-3 

3 

6 

Lower 

- 

Equal 

0 

- 

T: 

4 

1 

3 

n  : 

7 

3 

10 

P(x  <  T): 

0.7734 

0.3 

0.1710 

P(*  >  T): 

0.236ft 

0.3 

0.9201 

T  naedtd  to  Reject  Hoi 

6 

3 

7 

Decision  (alpha  =  .05  comptrtt  .10  reliability )i 

Table  M.16.  Sign  Test  for  Experiment  2,  Structure. 


PERSON 

FAC- MAN 

MANUAL 

OBJECTIVE 

COMPARISON 

SION  TEST 

SUBJECTIVE 

COMPARISON 

SION  TEST 

RELIABILITY 

SION  TEST 

3-1 

7 

7 

Equal 

0 

Equal 

0 

+ 

3-2 

6 

6 

Equal 

0 

Equal 

0 

+ 

3-3 

7 

6 

Higher 

+ 

Higher 

+ 

+ 

5-4 

5 

5 

Equal 

0 

Equal 

0 

+ 

5-5 

7 

6 

Higher 

+ 

Equal 

0 

- 

6-1 

7 

6 

Higher 

+ 

Lower 

- 

- 

6-2 

7 

6 

Higher 

+ 

Higher 

+ 

+ 

6-3 

7 

7 

Equal 

0 

Equal 

0 

+ 

6-4 

6 

5 

Higher 

+ 

Higher 

+ 

+ 

6-5 

6 

3 

Higher 

+ 

Lower 

- 

- 

T: 

6 

3 

7 

n : 

6 

5 

10 

P(x  <  T); 

1 

0.6125 

0.9433 

P(x  >  T) 

0 

0.1*75 

0.0547 

T  needed  to  Reject  Hoi 

5 

4 

7 

[Decision  (alphas. 05  compere* 

.10  reliability)! 

Reject  Ho 

Reject  Ho 

QUESTION:  Based  on  your  perceptions,  how  STRUCTURED  was  this  group 
process? 
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Appendix  N.  Facilitator  Comments 
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The  following  comments  were  obtained  from  the  facilitators  through  written  comments  provided 
on  the  Data  Collect' on  Forms  and  verbally  through  post-process  interviews. 

FACILITATOR.  SH 

TREATMtiV  I:  FAC-MAN  (non-GDSS) 

GROUP:  1 
DAY:  2 

CAREER  FIELD:  Manufacturing  Engineering 

1.  This  group  feels  that  there  are  not  too  many  true  manufacturing  engineers  left  in  the  USAF. 
These  professionals  are  now  process  engineers. 

2.  One  subject  did  not  show  much  interest  in  anything. 

3.  The  group  is  not  too  happy  nor  satisfied  with  the  task  definitions. 

4.  Two  subjects  dominate  the  discussion.  One  other  subject  is  not  so  interested,  and  another  only 
gives  quick,  crisp  answers. 

5.  The  group  feels  that  they  are  voting  for  someone  who  does  not  exist. 

FACILITATOR:  SH 

TREATMENT:  FAC-MAN  (non-GDSS) 

GROUP:  1 
DAY:  2 

CAREER  FIELD:  Toxicologist 

1.  Only  two  subjects  know  anything  about  the  subject  matter. 

FACILITATOR:  SH 

TREATMENT:  FAC-MAN  (non-GDSS) 

GROUP:  2 
DAY:  1 

CAREER  FIELD:  Ground  Safety 

1.  Two  individuals  were  very  influential. 

FACILITATOR:  SH 

TREATMENT:  FAC-MAN  (non-GDSS) 

GROUP:  2 
DAY:  1 

CAREER  FIELD.  System  Safety 

1 .  The  group  feels  that  the  proper  term  is  System  Safety  Engineer  rather  than  System  Safety. 

2.  There  are  many  doubts  on  the  wording  of  the  tasks. 

3.  Subjects  constantly  asked  for  scores  of  previous  tasks. 
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FACILITATOR:  GR 
TREATMENT:  GDSS 
GROUP:  2 
DAY:  2 

CAREER  FIELD:  Industrial  Shop  Worker 

1.  The  group  discussed  the  task  meanings  prior  to  the  initial  vote.  They  did  not  discuss  the  task  in 
relation  to  the  career  field  being  considered. 

2.  The  members  had  more  problems  casting  votes. 

3.  Member  2-4  departed  early  to  catch  an  airplane.  Resulted  in  missing  last  interval.  Interval  time 
before  departure  was  practically  identical  to  the  interval  following  departure,  13.67  and  13.58 
minutes,  respectively. 

FACILITATOR.  GR 
TREATMENT:  GDSS 
GROUP:  2 
DAY:  2 

CAREER  FIELD:  Industrial  Shop  Supervisor 

1.  The  voting  process  was  abandoned  for  expediency  purposes.  Votes  were  cast  across  all  tasks 
simultaneously  with  resolution  of  only  those  not  reaching  consensus. 

2.  One  member  stood  up  to  address  the  group  on  two  occasions  in  order  to  emphasize  a  point.  The 
member  stood  up  to  speak  over  the  computer  terminals  which  were  physically  a  communications 
barrier. 

FACILITATOR:  TE 
TREATMENT:  GDSS 
GROUP:  3 
DAY:  1 

CAREER  FIELD:  Fuels 

1 .  There  is  no  fuels  professional  represented  in  this  group. 

2.  Group  members  did  not  often  understand  the  tasks.  Discussion  of  task  definitions,  not  the  career 
field,  often  occurred  at  the  beginning  of  the  voting  process. 

FACILITATOR:  TE 
TREATMENT:  GDSS 
GROUP:  3 
DAY:  1 

CAREER  FIELD:  Plating 

1.  The  history  about  previous  careers  affects  consistency. 

FACILITATOR.  TE 
TREATMENT:  GDSS 
GROUP:  3 
DAY:  2 

CAREER  FIELD:  Maintenance  QA/QC 
1.  There  is  no  such  career  field,  but  a  mix  of  specialties. 
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FACILITATOR:  SI 

TREATMENT:  FAC-MAN  (non-GDSS) 

GROUP:  4 
DAY:  1 

CAREER  FIELD:  Real  Estate 

1 .  The  group  :s  relying  strongly  on  the  opinion  of  the  real  estate  professional  in  the  group. 

2.  Many  times  on  the  initial  or  second  vote,  members  hesitated  in  voting  until  the  real  estate 
professional  cast  his  vote.  Although  the  facilitator  discouraged  this  practice,  the  real  estate 
professional  was  very  influential  on  the  second  vote.  This  may  have  led  to  quicker  consensus  for 
this  group  with  this  career  field. 

FACILITATOR:  SI 

TREATMENT:  FAC-MAN  (non-GDSS) 

GROUP:  4 
DAY:  1 

CAREER  FIELD:  Contract  Inspector 

1.  The  group  interpreted  this  career  field  to  be  a  technician/engineer  assisting  the  Procurement 
Contracting  Officer  in  quality  assurance  (i.e.,  construction  or  manufacturing  plant  floor  inspector). 

2.  Some  small  discussion  occurred  before  some  first  votes  in  order  to  clarify  the  task  (no  value 
opinions  offered). 

FACILITATOR:  EM 
TREATMENT:  FAC-MAN  (non-GDSS) 

GROUP:  4 
DAY.  2 

CAREER  FIELD:  Procurement 

1.  Procurement  and  contracting  are  the  same  career  field  (contracting  is  the  old  terminology).  The 
group  will  not  vote  on  contracting  as  a  separate  career  field. 

2.  An  additional  person  (six  total)  participated  in  this  group’s  discussions,  but  did  not  vote  nor  fill 
out  any  questionnaires.  This  individual,  the  supervisor  for  one  group  member,  wanted  to  sit  in  on 
the  group’s  discussions  and  had  limited  influence  over  the  group’s  discussion  and  the  decision 
making  process. 

3.  This  was  an  excellent  group.  The  members  were  intelligent,  well-disciplined,  and  they  exhibited 
mature  adult  behavior.  The  discussions  were  open  and  frank. 

FACILITATOR:  EM 
TREATMENT:  FAC-MAN  (non-GDSS) 

GROUP.  5 
DAY:  1 

CAREER  FIELD:  Morale,  Welfare,  and  Recreation 
1.  There  are  three  MWR  professionals  in  this  group. 
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FACILITATOR:  EM 
TREATMENT:  FAC-MAN  (non-GDSS) 

GROUP:  5 
DAY:  1 

CAREER  FIELD:  Army,  Air  Force  Exchange  Service 

1.  There  weie  no  AAFES  representatives  in  this  group.  Discussions  did  not  proceed  very  far,  since 
the  group  had  no  expertise  in  this  area.  This  may  have  led  to  more  rapid  consensus,  and  did 
result  in  full  consensus  on  all  tasks. 

2.  The  air  conditioner  came  on  and  stayed  on  during  this  career  field.  By  the  end  of  this  session,  the 
members  were  cold.  This  factor  probably  contributed  to  more  rapidly  decision  making. 

3.  Since  AAFES  is  its  own  agency,  the  group  members  felt  that  this  career  field  required  higher 
training  levels,  frequently  level  five. 

FACILITATOR:  SI 

TREATMENT:  FAC-MAN  (non-GDSS) 

GROUP:  6 
DAY.  2 

CAREER  FIELD:  Vehicle  Maintenance 

1.  One  group  member  (who  participated  in  an  non-facilitated  process  on  Day  1)  did  not  like  the 
facilitated,  structured  process.  He  wanted  to  eliminate  the  first  vote  and  begin  with  a  discussion 
session.  The  group  decided  to  stay  with  the  three- vote  process. 

2.  The  group  spent  a  great  deal  of  discussion  to  clarify  tasks  and  to  clarify  the  5-point  rating  scale 
(training  levels). 

3.  The  group  had  trouble  distinguishing  between  level  four  and  five  ratings. 

FACILITATOR:  SI 

TREATMENT:  FAC-MAN  (non-GDSS) 

GROUP:  6 
DAY:  2 

CAREER  FIELD:  Waste  Treatment  -  Hazardous  Material 

1.  The  group  spent  10  minutes  (not  recorded)  of  discussion  prior  to  start  of  the  voting  process  in 
order  to  define  the  career  field. 
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Appendix  O.  Participant  Comments 
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GROUP.  1 

TREATMENT:  GDSS 
DAY:  1 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Efficient  voting  process  (automated). 

2.  Opportunity  for  discussion  was  good. 

3.  Need  to  reach  consensus. 

4.  Well-organized  and  planned — ready  for  group. 

NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Three  vote  process  could  work  better  if  consensus  is  clear  goal.  Slow,  but  shares  understanding  of 
subject. 

2.  Grossly  time-consuming.  No  flexibility.  Not  consulted  on  process  beforehand — held  hostage  to  an 
AFIT  experiment  without  permission. 

3.  Took  too  long — over  2  1/2  hours  for  first  one.  Should  vote  all  at  once  then  come  back  and  discuss. 

4.  Took  too  much  time! 

GENERAL  COMMENTS: 

1.  Process  change  for  Process  Engineer — identify  similarities  between  Process  Engineer  and  Material 
Engineer,  then  deal  with  non-consensus  by  exception.  Narrative  process  by  facilitator  slowed  first 
process. 

2.  Second  part  (career  field)  was  much  better,  but  required  that  first  was  reviewed  using  other 
procedure. 

3.  We  improved  the  process  on  the  second  round  by  comparing  and  highlighting  differences  in  the 
two  engineers. 

4.  Process  of  doing  the  second  career  field:  I  think  that  the  process  was  as  good  or  better  than  the 
proposed  method,  since  the  two  fields  were  similar. 

5.  Overall,  not  bad.  Second  process  run  on  Process  Engineer  career  field:  we  used  Materials 
Engineer  as  baseline  due  to  similarity  to  Process  Engineer  and  then  addressed  only  those  items 
not  the  same  as  an  exception. 

GROUP:  1 

TREATMENT:  FAC-MAN  (non-GDSS) 

DAY:  2 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Cards  promote  team  building  and  consensus  and  understanding. 

2.  Much  quicker.  More  efficient  for  small  group.  Can  communicate  differences/resolve  faster. 

3.  Worked  well  with  software  and  well  as  with  cards.  The  card  system  provided  better  interaction, 
communication,  and  was  quicker. 

4.  More  efficient  today.  Better  communications  (e.g.,  faster). 

NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Computer  interferes  with  group  communication. 

2.  More  peer  pressure  for  consensus. 

3.  Computer  process  work  was  too  long. 
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4.  Uncertainty  over  several  of  the  tasks  as  to  what  they  meant. 

GENERAL  COMMENTS: 

1.  Day  2  also  had  better  understanding  of  issues  being  decided. 

2.  For  small  groups,  this  system  is  preferred.  Larger  groups  would  benefit  from  computer  software. 

3.  Manual  card  system  best  for  small  groups — more  timely  with  at  least  as  good  product. 

4.  Pencil/paper  method  (today)  faster  and  more  efficient  for  a  small  group  like  this. 

GROUP:  2 

TREATMENT:  FAC-MAN  (non-GDSS) 

DAY:  1 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1 .  Personally,  I  now  have  a  better  understanding  of  the  types  and  amounts  of  environmental  training 
needed. 

2.  Good  group,  excellent  leader. 

3.  This  was  fun. 

4.  Significant  ability  to  accommodate  varying  viewpoints. 

5.  Group  exchanged  ideas  and  worked  towards  consensus. 

NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  We  had  to  make  assumptions  of  “occupations.”  Not  consistent  (i.e.,  System  Safety  vs  System 
Safety  Engineer — Big  Difference). 

2.  Was  not  clear  with  definition  of  category  I,  II,  and  III  stuff. 

3.  Tendency  to  hurry  the  process. 

4.  Basic  definitions  were  in  doubt  and  had  to  be  resolved  by  opinion  rather  than  fact. 

GENERAL  COMMENTS: 

1.  Need  to  receive  feedback  on  overall  result. 
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GROUP:  2 

TREATMENT:  GDSS 
DAY:  2 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  The  facilitator  helped  by  repeating  the  task  several  times  with  the  type  of  worker  keeping  focus 
on  the  vote  at  hand. 

2.  Team  members  could  both  talk  about  details  and  could  listen — good  group  dynamics. 

3.  Easier  to  discriminate  due  to  subject  matter. 

4.  Due  to  use  of  the  computer,  easier  to  recall  own  position  and  the  group’s  overall  position — helped 
focus. 

NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  The  group  generally  wanted  to  vote  on  level  of  knowledge  required  doing  both  “worker”  and 
“supervisor”  at  the  same  time.  The  facilitator  would  not  agree  to  this  process.  I  strongly  believe 
we  could  have  saved  much  time  by  not  going  over  each  task  twice.  Otherwise,  other  areas  were 
good. 

2.  Tasks  were  not  defined  properly  for  their  intended  application. 

3.  Coffee  would  have  been  nice  considering  we  paid  for  it  yesterday. 

GENERAL  COMMENTS: 

1.  We  were  too  rushed  to  get  done  and  get  back.  The  facilitator  did  not  rush  us,  but  knowing  we 
were  missing  the  larger  group  discussion  caused  me  to  want  to  rush. 

GROUP:  3 

TREATMENT:  GDSS 
DAY:  1 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Understanding  was  basically  achieved  quickly,  making  the  task  at  hand  much  easier  to  accomplish. 

2.  We  openly  discussed  disagreements  and  used  our  experience  and  logic  in  answering  questions. 

3.  Good  interchange  of  ideas. 

NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1  Lack  of  two  AFSCs  was  difficult  and  the  answers  where  from  the  gut  level. 

2.  We  lacked  an  individual  in  Fuels. 

3.  Timing  of  class  limited,  probably  hurt  discussion  some.  We  recognized  this  early  and  put 
Maintenance  on  the  second  day. 
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GENERAL  COMMENTS: 


1.  I  liked  the  software  and  use  of  computers. 

2.  Computer  program  shows  other  career  field  scores  when  all  the  way  to  the  right — it  can  influence 
your  decision. 

3.  The  facilitator  kept  us  focused  and  interacted  when  necessary  to  overcome  conflicts. 

GROUP.  3 

TREATMENT:  GDSS 
DAY:  2 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  The  group  worked  quite  well  today  and  yesterday. 

2.  We  remained  focused.  We  agreed  easier  on  conflicts  and  communicated  more  freely. 

3.  Rare  to  get  five  people  who  can  work  together.  We  were  very  productive. 

NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  We  guessed  somewhat  on  the  Fuels  functional  area,  but  used  logic  and  open  discussion  to  resolve 
conflicts. 

2.  NOT! 

GENERAL  COMMENTS: 

1.  Good  facilitating,  and  prompts  by  the  technographer  kept  us  focused.  We  also  had  less  of  a  time 
constraint  on  second  day,  since  first  day  was  late  afternoon  after  a  long  day  of  sitting  in  the  larger 
group. 

2.  No  change.  Both  days  went  very  well.  Good  computer  program.  I  think  it  enhanced  our 
performance. 

GROUP:  4 

TREATMENT:  FAC-MAN  (non-GDSS) 

DAY:  1 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Good  crossfeed. 

2.  Each  member  was  able  to  present  their  ideas  and  be  honest  with  each  other. 

3.  Learning  experience.  Good! 

4.  Good  discussion  of  viewpoints. 

NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Notes  did  not  correspond  to  tasking. 

2.  Discussion  lead  by  more  aggressive  members.  Not  everyone  participated  equally. 

GENERAL  COMMENTS:  (None) 
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GROUP:  4 

TREATMENT:  FAC-MAN  (non-GDSS) 

DAY:  2 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  I  feel  each  member  added  valuable  input.  Their  ideas  and  thoughts  were  valid. 

2.  Everyone  willing  to  entertain  others  perspective. 

NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  I  didn’t  feel  there  were  any. 

2.  Understanding  and  definition  of  tasks  appears  to  be  a  limiting  factor  on  the  overall  success  of  this 
exercise. 

3.  It  would  have  helped  if  sheets  passed  out  on  the  first  day  that  indicated  the  tasks,  per  meeting  at 
Brooks,  were  in  order.  For  instance,  the  group  on  “Weapon  Systems  Augmentation”  contained 
numbers  50,  30,  49,  15,  18,  32,  26,  and  48.  If  they  were  in  order,  our  job  would  have  been 
accomplished  more  efficiently  and  faster. 

GENERAL  COMMENTS: 

1.  Second  day  a  better  understanding  of  where  people  are  coming  from. 

2.  It  was  easier  today,  since  we  had  been  though  the  process  twice. 

3.  I  think  things  were  more  objective  the  first  day,  since  everyone  knew  how  to  “play”  the  second 
day. 

4.  Ground  rules  were  clearer.  More  aware  of  other  people’s  nonverbal  communication  cues.  More 
understanding  of  other’s  thought  processes.  Definitions  of  environmental  tasks  were  already 
agreed  to. 

GROUP:  5 

TREATMENT:  FAC-MAN  (NON-GDSS) 

DAY:  1 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Good  discussion  of  differing  points  of  view  and  good  points  of  explaining  reasoning. 

2.  Very  congenial  group.  Task  oriented,  good  discussion  (persuasive),  and  good  listening. 

3.  Process  worked  well  with  solid  participation  by  all. 

NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Group  had  no  members  from  two  of  the  areas  discussed  (AAFES  and  DECA).  Still  some 
confusion  over  what  tasks  actually  meant. 

2.  Unsure  of  full  aspects  or  definition  of  some  tasks.  No  representative  in  second  career  field 
category  which  forced  group  to  make  assumptions. 

3.  Not  having  reps  from  two  of  the  agencies. 
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GENERAL  COMMENTS: 

1.  Some  task  descriptions  over-lapped  or  are  vague. 

2.  This  group  was  asked  to  rate  four  AFSCs,  but  there  were  not  representatives  for  two  AFSCs 
(AAFES  and  DECA).  Not  sure  we  were  accurate  in  rating  those  areas. 

3.  Well  facilitated.  Willingness  to  work  towards  the  good  of  the  Air  Force — focused! 

GROUP:  5 

TREATMENT:  Manual 
DAY:  2 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Same  as  provided  on  first  session  questionnaire.  Learned  more  about  process  evaluation  and 
about  other  areas  involvement. 

2.  Very  agreeable. 

3.  Well-organized  and  structured — team  effort. 

NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Really  needed  a  better  definition  of  what  tasks  involved,  scope,  and  application  intended  to  better 
assess  task  in  relation  to  area  (job). 

2.  Lack  of  knowledge  with  DECA  career  field.  Some  problems  with  task  definition  understandings. 

3.  Two  agencies  not  represented. 

GENERAL  COMMENTS: 

1.  Used  cards  (1-5)  first  day — liked  that  better  than  voice  voting.  Helped  in  reaching  consensus. 

2.  Yesterday’s  session  had  a  facilitator  and  more  energetic  discussions,  but  also  seemed  more 
stressful.  Learning  curve  involved  in  process  and  scoring  decisions. 

3.  Preferred  first  day’s  process — a  bit  more  time  consuming,  but  drives  more  individual  effort  prior 
to  team  effort/concentration. 

GROUP:  6 

TREATMENT:  Manual 
DAY:  1 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  The  group  members  worked  together,  listened  to  the  viewpoints  of  others,  and  were  flexible 
enough  to  modify  their  scoring  when  additional  data  surfaced. 

2.  Discussion  was  very  productive.  Everyone  listened  to  other  people’s  viewpoints  and  answers  were 
based  on  true  consensus. 

3.  Felt  group  was  well-balanced  in  terms  of  personality,  technical  background/experience. 

4.  Consensus  reached  quickly. 
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NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Nothing  really  negative  about  the  group  or  the  process. 

2.  This  group  lacks  working  level  expertise  in  the  areas  considered.  Answers  should  be  checked  with 
these  working  areas. 

3.  Tasks  were  too  vague  in  areas  such  as  Emergency  Response.  Functional  groups  (i.e.,  plating  shop) 
should  be  broken  into  three  levels  of  responsibility  (i.e.,  command,  supervisor,  worker). 

4.  Concerned  that  the  definitions/interpretations  used  by  the  group  for  both  jobs  and  tasks,  while 
applied  consistently  within  the  context  of  the  group  activity,  are  different  from  those  of  other 
groups,  and  will  make  aggregation  of  results  of  questionable  value. 

5.  Lack  of  understanding  of  issues. 

GENERAL  COMMENTS: 

1.  Issues  need  clarification  to  speed  process  up. 

GROUP:  6 

TREATMENT:  FAC-MAN  (non-GDSS) 

DAY:  2 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  First  vote  with  no  discussion  gave  a  real  indication  of  how  the  member  actually  felt  given  his/her 
knowledge  of  the  task  and  the  career  field. 

2.  All  tasks  are  considered  from  a  broad  view  given  the  variety  of  expertise  in  the  group,  giving,  in 
my  opinion,  credibility  to  the  process. 

3.  Consensus  may  have  been  reached  with  differing  understanding  in  first  vote.  Quality  of  overall 
effort  may  have  suffered. 

NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Initial  votes  led  to  the  minority  opinions  (votes)  being  automatically  defended  by  the  minority 
voter.  The  member  then  stood  their  ground,  to  a  certain  degree,  regardless  of  the  discussion — this 
psychological  outcome  is  probably  based  on  a  need  to  conceal  a  lack  of  knowledge  about  either 
the  task  or  the  target  group.  The  method  used  today  caused  more  conflicts  to  arise  and  the 
process  was  less  conducive  to  a  positive  result. 

2.  The  ability  to  talk  through  conflict  and  reach  consensus  was  hampered  by  the  structure  of  the 
voting  process.  There  was  a  lower  level  of  interaction  among  group  members  today.  The  formal 
voting  system  didn’t  work  as  well. 

3.  Too  much  pressure  to  compromise.  Time  crunch. 

GENERAL  COMMENTS: 

1.  I  preferred  discussing  the  task  prior  to  voting  as  we  did  the  first  day.  To  vote  first  generates,  as  a 
rule,  unusable  data  due  to  confusion  of  what  the  task  really  involves. 

2.  Use  of  number  cards  posed  somewhat  of  a  psychological  barrier,  in  that  it  formalized  our 
responses.  Over  time,  this  barrier  was  overcome  somewhat  (compensated  for)  as  we  became  more 
comfortable  with  the  mechanism. 

3.  More  discussion  of  issues,  because  of  not  being  aware  of  peoples  starting  position. 
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GROUP:  7 

TREATMENT:  Manual 
DAY:  1 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Very  organized  and  focused. 

2.  It  was  great! 

NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  None. 

2.  None. 

GENERAL  COMMENTS:  (None.) 

GROUP:  7 

TREATMENT:  Manual 
DAY:  2 

POSITIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE: 

1.  Good  team — level/range  of  expertise  was  beneficial. 

2.  Professional,  positive,  and  focused. 

3.  Excellent  group! 

NEGATIVE  COMMENTS  CONCERNING  GROUP’S  PROCESS  OR  EXPERIENCE:  (None.) 
GENERAL  COMMENTS: 

1.  Group  participation  and  problem-solving  capacity  was  excellent! 
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Appendix  P.  Glossary 
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AAFES 

Army  Air  Force  Exchange  Service 

AF 

Air  Force 

AFB 

Air  Force  Base 

AFIT 

Air  Force  Institute  of  Technology 

AL/HRG 

Armstrong  Laboratory/Human  Resources  Directorate/Logistics 
Research  Division 

DECA 

Defense  Commissary  Agency 

DP 

Disaster  Preparedness 

DSC 

Decision  Support  Center 

DSS 

Decision  Support  System 

Exp 

Experiment 

FAC-MAN 

Facilitated-Manual 

GCSS 

Group  Communications  Support  System 

GDSS 

Group  Decision  Support  System 

GSS 

Group  Support  System 

HAZMAT 

Hazardous  Material 

MIS 

Management  Information  Systems 

MTF 

Medical  Treatment  Facility 

NGT 

Nominal  Group  Technique 

PC 

Personal  Computer 

TDY 

Temporary  Duty 

USAF 

United  States  Air  Force 

VIF 

Variance  Inflation  Factor 
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real  managers  making  real  decisions.  This  research  was  conducted  by  the  Air  Force  Institute  of  Technology  with 
the  Air  Force  Armstrong  Laboratory  to  evaluate  the  efficacy  of  using  GDSS  technology  in  an  Air  Force  decision 
making  environment.  This  study  compared  facilitated  GDSS,  facilitated  manual,  and  unfacilitated  manual 
decision  making  processes.  This  study  focused  on  performance  and  secondary  outcomes  resulting  from  GDSS 
effects  versus  process  structure  effects.  The  research  evaluated  two  performance  outcomes:  GDSS  efficiency  in 
terms  of  decision  speed  and  effectiveness  in  terms  of  consensus.  Post-process  questionnaires  were  used  to  compare 
groups  with  respect  to  the  participant’s  perceptions.  The  study  concluded  that  GDSS,  process  structure,  and 
group  dynamics  impacted  performance.  GDSS  groups  experienced  lower  decision  speeds  and  consensus  compared 
to  facilitated  manual  groups  which,  in  turn,  experienced  lower  decision  speeds  and  consensus  compared  to 
unfacilitated  manual  groups.  Participants  perceived  the  GDSS  process  as  more  difficult  than  the  facilitated 
manual  process.  Likewise,  participants  perceived  the  facilitated  manual  process  as  more  difficult,  more  structured, 
and  producing  more  conflict  than  the  unfacilitated  manual  process.  These  findings  are  consistent  with  other  field 
studies  which  concluded  that  small  groups  or  groups  performing  less  complex  choice-type  tasks  may  not  benefit 
from  use  of  a  GDSS. 
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